2024-06-02T15:13:27.203 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-06-02T15:13:27.203 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920 --verbose --name teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/misc}} -- /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/orig.config.yaml 2024-06-02T15:13:27.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-02T15:13:27.346 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920 branch: main description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7720920' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b7328fed0e2e9ab66b09f8134f3cebfc71214504 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: b7328fed0e2e9ab66b09f8134f3cebfc71214504 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7aec00165b4b073a25396fd0279151d2b2fa23fe owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6827 sha1: b7328fed0e2e9ab66b09f8134f3cebfc71214504 sleep_before_teardown: 0 subset: 382/512 suite: fs suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7aec00165b4b073a25396fd0279151d2b2fa23fe syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNkza4Ngzy196EduGFTAaM2O69K2N8hZ74lteuavPkIrp9OyFaPQ6wQ3yrry6agshCKC1L/7Mec9PI6HD7+Ksj4= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Cl+yFmTvyTiW1obry6bw8qN8+f9kzUfly5YUQAbVv29NnrOteHgDCeTgGFN5/q/7YHQD7y2ofzc+co0rbOT8= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgVGGvuPhM6WEShO+JEwhXBZW0FqHI3zaWXLGezVV5ItgfcS2qvAvyKq3dgHB6XOsQUiCL6CekcM6+aGXmMcjw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-22_20:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-02T15:13:27.346 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa; will attempt to use it 2024-06-02T15:13:27.347 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks 2024-06-02T15:13:27.347 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-02T15:13:27.349 INFO:teuthology.task.internal:Checking packages... 2024-06-02T15:13:27.369 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b7328fed0e2e9ab66b09f8134f3cebfc71214504' 2024-06-02T15:13:27.369 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-02T15:13:27.369 INFO:teuthology.packaging:ref: None 2024-06-02T15:13:27.369 INFO:teuthology.packaging:tag: None 2024-06-02T15:13:27.369 INFO:teuthology.packaging:branch: main 2024-06-02T15:13:27.369 INFO:teuthology.packaging:sha1: b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:13:27.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-06-02T15:13:27.687 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-4019.g17c6b5a1 2024-06-02T15:13:27.689 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-02T15:13:27.696 INFO:teuthology.task.internal:no buildpackages task found 2024-06-02T15:13:27.696 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-02T15:13:27.755 INFO:teuthology.task.internal:Saving configuration 2024-06-02T15:13:27.774 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-02T15:13:27.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-02T15:13:27.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 15:07:49.779351', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNkza4Ngzy196EduGFTAaM2O69K2N8hZ74lteuavPkIrp9OyFaPQ6wQ3yrry6agshCKC1L/7Mec9PI6HD7+Ksj4='} 2024-06-02T15:13:27.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 15:07:49.777643', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP+Cl+yFmTvyTiW1obry6bw8qN8+f9kzUfly5YUQAbVv29NnrOteHgDCeTgGFN5/q/7YHQD7y2ofzc+co0rbOT8='} 2024-06-02T15:13:27.849 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 15:07:49.774814', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAgVGGvuPhM6WEShO+JEwhXBZW0FqHI3zaWXLGezVV5ItgfcS2qvAvyKq3dgHB6XOsQUiCL6CekcM6+aGXmMcjw='} 2024-06-02T15:13:27.849 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-02T15:13:27.856 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-02T15:13:27.856 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-02T15:13:27.856 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-02T15:13:27.856 INFO:teuthology.run_tasks:Running task console_log... 2024-06-02T15:13:27.993 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff8d76fbe0>, signals=[15]) 2024-06-02T15:13:27.993 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-02T15:13:27.999 INFO:teuthology.task.internal:Opening connections... 2024-06-02T15:13:28.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:13:28.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:13:28.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-06-02T15:13:28.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:13:28.146 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-06-02T15:13:28.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:13:28.219 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-02T15:13:28.225 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-06-02T15:13:28.243 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-06-02T15:13:28.243 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-06-02T15:13:28.300 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T15:13:28.301 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T15:13:28.301 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T15:13:28.301 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-06-02T15:13:28.324 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-06-02T15:13:28.342 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-06-02T15:13:28.342 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T15:13:28.399 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-06-02T15:13:28.400 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T15:13:28.400 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T15:13:28.400 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T15:13:28.400 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-06-02T15:13:28.424 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-06-02T15:13:28.442 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-06-02T15:13:28.442 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-06-02T15:13:28.499 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-06-02T15:13:28.500 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T15:13:28.501 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T15:13:28.501 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T15:13:28.501 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-06-02T15:13:28.516 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-02T15:13:28.526 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-02T15:13:28.532 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-02T15:13:28.532 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-06-02T15:13:28.535 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-06-02T15:13:28.537 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-06-02T15:13:28.556 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-02T15:13:28.564 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-02T15:13:28.564 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-06-02T15:13:28.591 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-06-02T15:13:28.595 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-06-02T15:13:28.687 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-02T15:13:28.764 INFO:teuthology.run_tasks:Running task kernel... 2024-06-02T15:13:28.778 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-06-02T15:13:28.779 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-06-02T15:13:28.779 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-06-02T15:13:28.779 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T15:13:28.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:13:28.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-06-02T15:13:29.056 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-02T15:13:29.056 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-06-02T15:13:29.056 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-06-02T15:13:29.074 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-06-02T15:13:29.074 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-02T15:13:29.074 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-06-02T15:13:29.074 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-06-02T15:13:31.655 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-02T15:13:54.795 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 887 kB/s | 19 MB 00:22 2024-06-02T15:13:58.887 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-02T15:14:00.247 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 17 kB 00:00 2024-06-02T15:14:01.366 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-06-02T15:14:06.534 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-06-02T15:14:08.683 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-06-02T15:14:08.684 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-06-02T15:14:08.684 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-06-02T15:14:08.777 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-02T15:14:08.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-06-02T15:14:09.000 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-06-02T15:14:09.001 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-06-02T15:14:09.003 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-06-02T15:14:47.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02 15:14:47 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-06-02T15:14:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-06-02 15:14:47-- 2024-06-02T15:14:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 38s 2024-06-02T15:14:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 340M in 38s (9.06 MB/s) 2024-06-02T15:14:47.032 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-06-02T15:14:47.032 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-06-02T15:14:49.137 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-06-02T15:14:49.235 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-06-02T15:14:49.356 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-06-02T15:14:57.332 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-06-02T15:15:39.520 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-06-02T15:15:39.521 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-06-02T15:15:39.618 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/config-6.9.0-gb9459a26b088 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-06-02T15:15:39.619 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-06-02T15:15:39.620 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-06-02T15:15:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-06-02T15:15:39.622 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-06-02T15:15:39.623 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-06-02T15:15:39.624 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-06-02T15:15:39.625 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-06-02T15:15:39.626 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-06-02T15:15:39.627 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-06-02T15:15:39.628 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-06-02T15:15:39.629 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-06-02T15:15:39.630 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-06-02T15:15:39.631 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-06-02T15:15:39.632 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-06-02T15:15:39.633 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-06-02T15:15:39.634 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-06-02T15:15:39.635 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-06-02T15:15:39.636 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-06-02T15:15:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-06-02T15:15:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-06-02T15:15:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-06-02T15:15:39.640 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-06-02T15:15:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-06-02T15:15:39.642 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-06-02T15:15:39.643 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-06-02T15:15:39.644 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-06-02T15:15:39.644 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-06-02T15:15:39.644 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-06-02T15:15:39.644 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-06-02T15:15:39.645 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-06-02T15:15:39.645 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-06-02T15:15:39.646 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-06-02T15:15:39.646 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-06-02T15:15:39.646 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-06-02T15:15:39.646 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-06-02T15:15:39.646 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-06-02T15:15:39.647 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-06-02T15:15:39.647 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-06-02T15:15:39.647 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-06-02T15:15:39.647 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-06-02T15:15:39.648 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-06-02T15:15:39.648 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-06-02T15:15:39.648 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-06-02T15:15:39.648 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-06-02T15:15:39.648 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-06-02T15:15:39.649 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-06-02T15:15:39.649 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-06-02T15:15:39.649 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-06-02T15:15:39.649 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-06-02T15:15:39.649 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-06-02T15:15:39.650 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-06-02T15:15:39.650 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-06-02T15:15:39.650 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-06-02T15:15:39.651 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-06-02T15:15:39.651 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-06-02T15:15:39.651 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-06-02T15:15:39.651 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-06-02T15:15:39.652 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-06-02T15:15:39.653 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-06-02T15:15:39.653 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-06-02T15:15:39.653 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-06-02T15:15:39.653 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-06-02T15:15:39.653 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-06-02T15:15:39.654 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-06-02T15:15:39.654 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-06-02T15:15:39.654 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-06-02T15:15:39.654 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-06-02T15:15:39.655 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-06-02T15:15:39.655 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-06-02T15:15:39.655 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-06-02T15:15:39.655 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-06-02T15:15:39.656 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-06-02T15:15:39.656 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-06-02T15:15:39.656 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-06-02T15:15:39.656 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-06-02T15:15:39.657 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-06-02T15:15:39.657 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-06-02T15:15:39.657 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-06-02T15:15:39.657 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-06-02T15:15:39.657 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-06-02T15:15:39.658 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-06-02T15:15:39.658 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-06-02T15:15:39.658 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-06-02T15:15:39.658 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-06-02T15:15:39.659 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-06-02T15:15:39.659 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-06-02T15:15:39.659 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-06-02T15:15:39.659 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-06-02T15:15:39.660 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-06-02T15:15:39.660 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-06-02T15:15:39.660 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-06-02T15:15:39.660 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-06-02T15:15:39.660 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-06-02T15:15:39.661 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-06-02T15:15:39.661 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-06-02T15:15:39.661 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-06-02T15:15:39.662 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-06-02T15:15:39.662 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-06-02T15:15:39.662 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-06-02T15:15:39.662 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-06-02T15:15:39.662 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-06-02T15:15:39.663 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-06-02T15:15:39.663 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-06-02T15:15:39.663 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-06-02T15:15:39.663 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-06-02T15:15:39.663 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-06-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-06-02T15:15:39.665 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-06-02T15:15:39.665 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-06-02T15:15:39.665 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-06-02T15:15:39.665 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-06-02T15:15:39.665 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-06-02T15:15:39.666 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-06-02T15:15:39.667 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-06-02T15:15:39.668 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-06-02T15:15:39.668 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-06-02T15:15:39.668 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-06-02T15:15:39.668 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-06-02T15:15:39.669 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-06-02T15:15:39.670 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-06-02T15:15:39.670 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-06-02T15:15:39.670 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-06-02T15:15:39.670 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-06-02T15:15:39.671 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-06-02T15:15:39.672 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-06-02T15:15:39.672 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-06-02T15:15:39.672 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-06-02T15:15:39.672 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-06-02T15:15:39.673 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-06-02T15:15:39.673 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-06-02T15:15:39.673 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-06-02T15:15:39.673 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-06-02T15:15:39.673 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-06-02T15:15:39.674 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-06-02T15:15:39.674 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-06-02T15:15:39.674 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-06-02T15:15:39.674 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-06-02T15:15:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-06-02T15:15:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-06-02T15:15:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-06-02T15:15:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-06-02T15:15:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-06-02T15:15:39.676 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-06-02T15:15:39.676 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-06-02T15:15:39.676 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-06-02T15:15:39.676 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-06-02T15:15:39.677 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-06-02T15:15:39.677 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-06-02T15:15:39.677 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-06-02T15:15:39.677 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-06-02T15:15:39.678 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-06-02T15:15:39.678 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-06-02T15:15:39.678 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-06-02T15:15:39.679 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-06-02T15:15:39.679 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-06-02T15:15:39.680 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-06-02T15:15:39.681 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-06-02T15:15:39.682 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-06-02T15:15:39.683 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-06-02T15:15:39.684 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-06-02T15:15:39.685 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-06-02T15:15:39.686 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-06-02T15:15:39.687 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-06-02T15:15:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-06-02T15:15:39.689 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-06-02T15:15:39.690 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-06-02T15:15:39.691 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-06-02T15:15:39.692 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-06-02T15:15:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-06-02T15:15:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-06-02T15:15:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-06-02T15:15:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-06-02T15:15:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-06-02T15:15:39.698 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-06-02T15:15:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-06-02T15:15:39.700 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-06-02T15:15:39.701 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-06-02T15:15:39.702 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-06-02T15:15:39.703 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-06-02T15:15:39.704 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-06-02T15:15:39.705 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-06-02T15:15:39.706 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-06-02T15:15:39.707 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-06-02T15:15:39.708 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-06-02T15:15:39.709 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-06-02T15:15:39.710 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-06-02T15:15:39.711 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-06-02T15:15:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-06-02T15:15:39.713 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-06-02T15:15:39.714 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-06-02T15:15:39.715 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-06-02T15:15:39.716 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-06-02T15:15:39.717 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-06-02T15:15:39.718 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-06-02T15:15:39.719 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-06-02T15:15:39.720 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-06-02T15:15:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-06-02T15:15:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-06-02T15:15:39.723 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-06-02T15:15:39.724 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-06-02T15:15:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-06-02T15:15:39.726 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-06-02T15:15:39.727 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-06-02T15:15:39.728 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-06-02T15:15:39.729 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-06-02T15:15:39.730 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-06-02T15:15:39.731 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-06-02T15:15:39.732 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-06-02T15:15:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-06-02T15:15:39.734 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-06-02T15:15:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-06-02T15:15:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-06-02T15:15:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-06-02T15:15:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-06-02T15:15:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-06-02T15:15:39.740 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-06-02T15:15:39.741 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-06-02T15:15:39.742 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-06-02T15:15:39.743 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-06-02T15:15:39.744 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-06-02T15:15:39.745 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-06-02T15:15:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-06-02T15:15:39.747 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-06-02T15:15:39.748 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-06-02T15:15:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-06-02T15:15:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-06-02T15:15:39.751 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-06-02T15:15:39.752 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-06-02T15:15:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-06-02T15:15:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-06-02T15:15:39.755 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-06-02T15:15:39.756 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-06-02T15:15:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-06-02T15:15:39.758 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-06-02T15:15:39.759 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-06-02T15:15:39.760 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-06-02T15:15:39.761 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-06-02T15:15:39.762 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-06-02T15:15:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-06-02T15:15:39.764 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-06-02T15:15:39.765 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-06-02T15:15:39.766 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-06-02T15:15:39.767 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-06-02T15:15:39.768 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-06-02T15:15:39.769 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-06-02T15:15:39.770 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-06-02T15:15:39.771 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-06-02T15:15:39.772 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-06-02T15:15:39.772 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-06-02T15:15:39.772 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-06-02T15:15:39.772 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-06-02T15:15:39.772 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-06-02T15:15:39.773 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-06-02T15:15:39.773 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-06-02T15:15:39.773 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-06-02T15:15:39.773 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-06-02T15:15:39.774 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-06-02T15:15:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-06-02T15:15:39.776 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-06-02T15:15:39.777 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-06-02T15:15:39.778 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-06-02T15:15:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-06-02T15:15:39.780 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-06-02T15:15:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-06-02T15:15:39.782 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-06-02T15:15:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-06-02T15:15:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-06-02T15:15:39.785 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-06-02T15:15:39.786 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-06-02T15:15:39.787 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-06-02T15:15:39.788 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-06-02T15:15:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-06-02T15:15:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-06-02T15:15:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-06-02T15:15:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-06-02T15:15:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-06-02T15:15:39.794 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-06-02T15:15:39.795 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-06-02T15:15:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-06-02T15:15:39.797 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-06-02T15:15:39.798 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-06-02T15:15:39.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-06-02T15:15:39.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-06-02T15:15:39.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-06-02T15:15:39.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-02T15:15:39.805 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-06-02T15:15:39.806 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-06-02T15:15:39.831 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-06-02T15:15:39.831 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-02T15:15:39.831 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-06-02T15:15:39.832 INFO:teuthology.orchestra.run.smithi090.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-06-02T15:15:39.833 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-06-02T15:15:39.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-06-02T15:15:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-02T15:15:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-02T15:15:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-06-02T15:15:39.912 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:15:39.912 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-06-02T15:15:39.913 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-02T15:15:39.913 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.9.0-gb9459a26b088 2024-06-02T15:15:39.913 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-02T15:15:40.489 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-06-02T15:15:42.313 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-02T15:15:42.337 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-06-02T15:15:42.340 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-06-02T15:15:42.375 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.ecmkd2jj8c 2024-06-02T15:15:42.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ecmkd2jj8c 2024-06-02T15:15:42.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.ecmkd2jj8c 2024-06-02T15:15:42.599 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.ecmkd2jj8c is 6KB 2024-06-02T15:15:42.648 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.ecmkd2jj8c 2024-06-02T15:15:42.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-06-02T15:15:42.723 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-02T15:15:42.723 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-02T15:15:42.724 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-02T15:15:42.724 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-06-02T15:15:42.725 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-06-02T15:15:42.871 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-06-02T15:16:12.874 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-06-02T15:16:12.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:16:12.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:16:31.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-06-02T15:16:40.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:16:40.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:16:43.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-06-02T15:16:55.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:16:55.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:16:58.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-06-02T15:17:13.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:17:13.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:18:13.531 DEBUG:teuthology.orchestra.remote:timed out 2024-06-02T15:18:31.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:18:31.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:18:32.144 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-06-02T15:18:32.864 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:18:32.864 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-06-02T15:18:32.864 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-06-02T15:18:32.917 INFO:teuthology.orchestra.run.smithi090.stdout:6.9.0-gb9459a26b088 2024-06-02T15:18:32.918 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-06-02T15:18:32.918 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-02T15:18:32.918 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-02T15:18:33.919 DEBUG:teuthology.parallel:result is None 2024-06-02T15:18:33.919 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-02T15:18:33.929 INFO:teuthology.task.internal:Creating test directory... 2024-06-02T15:18:33.929 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T15:18:33.937 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T15:18:33.939 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T15:18:33.987 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-02T15:18:33.994 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-02T15:18:34.004 INFO:teuthology.task.internal:Creating archive directory... 2024-06-02T15:18:34.004 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T15:18:34.011 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T15:18:34.014 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T15:18:34.203 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-02T15:18:34.210 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-02T15:18:34.210 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T15:18:34.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:18:34.227 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T15:18:34.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:18:34.244 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T15:18:34.251 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T15:18:34.270 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T15:18:34.317 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.324 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.337 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.338 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.406 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.461 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T15:18:34.465 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-02T15:18:34.473 INFO:teuthology.task.internal:Configuring sudo... 2024-06-02T15:18:34.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T15:18:34.480 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T15:18:34.482 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T15:18:34.555 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-02T15:18:34.565 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-02T15:18:34.565 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T15:18:34.572 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T15:18:34.574 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T15:18:34.618 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:34.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:34.797 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:34.878 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:34.963 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:18:34.963 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T15:18:35.045 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:35.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:35.152 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:35.194 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:35.235 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:18:35.236 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T15:18:35.300 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:35.344 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T15:18:35.407 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:35.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T15:18:35.492 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:18:35.492 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T15:18:35.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-06-02T15:18:35.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-06-02T15:18:35.568 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-06-02T15:18:35.600 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T15:18:35.632 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T15:18:35.671 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T15:18:36.213 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-02T15:18:36.220 INFO:teuthology.task.internal:Starting timer... 2024-06-02T15:18:36.221 INFO:teuthology.run_tasks:Running task pcp... 2024-06-02T15:18:36.240 INFO:teuthology.run_tasks:Running task selinux... 2024-06-02T15:18:36.249 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-02T15:18:36.249 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-06-02T15:18:36.357 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-06-02T15:18:36.362 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-06-02T15:18:36.436 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-06-02T15:18:36.438 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-06-02T15:18:36.525 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-06-02T15:18:36.527 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-02T15:18:36.527 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-06-02T15:18:36.584 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-06-02T15:18:36.584 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-06-02T15:18:36.622 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-06-02T15:18:36.622 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-06-02T15:18:36.644 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-06-02T15:18:36.645 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-06-02T15:18:36.645 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T15:18:36.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:18:36.716 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T15:18:36.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:18:36.747 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T15:18:36.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T15:18:36.772 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-02T15:18:36.772 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-06-02T15:18:36.845 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-06-02T15:18:36.872 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-06-02T15:18:36.896 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-02T15:18:36.905 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-02T15:18:36.917 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-02T15:18:36.918 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi120.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-02T15:25:03.377 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-06-02T15:25:03.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:25:03.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:25:03.494 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-06-02T15:25:03.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-06-02T15:25:03.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-06-02T15:25:03.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:25:03.716 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-06-02T15:25:03.794 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-06-02T15:25:03.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-02T15:25:03.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T15:25:03.868 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-06-02T15:25:03.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-06-02T15:25:03.954 INFO:teuthology.run_tasks:Running task clock... 2024-06-02T15:25:03.965 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-02T15:25:03.965 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T15:25:03.966 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T15:25:03.973 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T15:25:03.974 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T15:25:03.978 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T15:25:03.978 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T15:25:04.005 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T15:25:04.020 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T15:25:04.021 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T15:25:04.034 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T15:25:04.044 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-06-02T15:25:04.055 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-06-02T15:25:04.056 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-06-02T15:25:04.063 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T15:25:04.066 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-06-02T15:25:04.069 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T15:25:04.079 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T15:25:04.082 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T15:25:04.092 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T15:25:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T15:25:04.138 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.141 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.154 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.170 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-06-02T15:25:04.200 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-06-02T15:25:04.236 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T15:25:04.277 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T15:25:04.399 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.407 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T15:25:04.408 INFO:teuthology.run_tasks:Running task install... 2024-06-02T15:25:04.417 DEBUG:teuthology.task.install:project ceph 2024-06-02T15:25:04.417 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504'}} 2024-06-02T15:25:04.418 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504'} 2024-06-02T15:25:04.418 INFO:teuthology.task.install:Using flavor: default 2024-06-02T15:25:04.424 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-02T15:25:04.425 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-02T15:25:04.425 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504', 'tag': None, 'wait_for_package': False} 2024-06-02T15:25:04.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:25:04.428 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504', 'tag': None, 'wait_for_package': False} 2024-06-02T15:25:04.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:25:04.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504', 'tag': None, 'wait_for_package': False} 2024-06-02T15:25:04.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:25:04.659 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/ 2024-06-02T15:25:04.659 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3882.gb7328fed 2024-06-02T15:25:04.662 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/ 2024-06-02T15:25:04.663 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3882.gb7328fed 2024-06-02T15:25:04.666 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/ 2024-06-02T15:25:04.666 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3882.gb7328fed 2024-06-02T15:25:04.900 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T15:25:04.900 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:25:04.900 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T15:25:04.902 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T15:25:04.903 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:25:04.903 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T15:25:04.928 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T15:25:04.928 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b7328fed0e2e9ab66b09f8134f3cebfc71214504/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T15:25:04.929 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T15:25:04.929 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b7328fed0e2e9ab66b09f8134f3cebfc71214504/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T15:25:04.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T15:25:04.996 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T15:25:05.013 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T15:25:05.013 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:25:05.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T15:25:05.067 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T15:25:05.072 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T15:25:05.102 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T15:25:05.103 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b7328fed0e2e9ab66b09f8134f3cebfc71214504/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T15:25:05.129 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-06-02T15:25:05.131 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-06-02T15:25:05.135 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-06-02T15:25:05.137 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-06-02T15:25:05.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T15:25:05.343 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T15:25:05.353 INFO:teuthology.orchestra.run.smithi120.stdout:45 files removed 2024-06-02T15:25:05.360 INFO:teuthology.orchestra.run.smithi135.stdout:45 files removed 2024-06-02T15:25:05.380 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T15:25:05.387 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T15:25:05.425 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-06-02T15:25:05.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-06-02T15:25:05.739 INFO:teuthology.orchestra.run.smithi090.stdout:45 files removed 2024-06-02T15:25:05.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T15:25:06.466 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 102 kB/s | 84 kB 00:00 2024-06-02T15:25:06.570 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 90 kB/s | 84 kB 00:00 2024-06-02T15:25:07.210 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 83 kB/s | 84 kB 00:01 2024-06-02T15:25:07.234 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-06-02T15:25:07.337 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-06-02T15:25:08.026 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-06-02T15:25:08.129 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-06-02T15:25:08.235 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-06-02T15:25:08.554 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.1 MB 00:00 2024-06-02T15:25:08.877 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.1 MB 00:00 2024-06-02T15:25:09.261 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-06-02T15:25:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-02T15:25:10.636 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-02T15:25:12.061 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-06-02T15:25:13.331 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-06-02T15:25:14.887 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-06-02T15:25:16.547 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-02T15:25:16.630 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 17 kB 00:00 2024-06-02T15:25:18.085 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-02T15:25:19.315 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 17 kB 00:01 2024-06-02T15:25:19.973 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-06-02T15:25:20.767 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-06-02T15:25:21.466 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-06-02T15:25:22.409 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-06-02T15:25:23.358 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-02T15:25:25.676 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:25.676 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:25.687 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T15:25:25.688 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T15:25:25.689 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T15:25:25.690 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T15:25:25.691 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T15:25:25.694 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T15:25:25.695 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T15:25:25.758 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T15:25:25.776 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 6.6 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 890 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 145 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 1.5 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 143 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 3.5 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 7.4 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 49 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 11 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 49 M 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 273 k 2024-06-02T15:25:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 763 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 225 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 27 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 18 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 32 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 747 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 127 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T15:25:25.778 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 161 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 320 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 300 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 100 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 85 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 170 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T15:25:25.779 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.4 M 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T15:25:25.780 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 21 M 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 34 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 2.4 M 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 274 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 4.7 M 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 17 M 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 16 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 25 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T15:25:25.781 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 163 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 501 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T15:25:25.782 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T15:25:25.783 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 45 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 135 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T15:25:25.784 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T15:25:25.785 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T15:25:25.786 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T15:25:25.787 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T15:25:25.788 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout:======================================================================================= 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 11 Packages 2024-06-02T15:25:25.789 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:25:25.790 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 227 M 2024-06-02T15:25:25.790 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-06-02T15:25:26.351 INFO:teuthology.orchestra.run.smithi135.stdout:(1/159): ceph-19.0.0-3882.gb7328fed.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-06-02T15:25:26.777 INFO:teuthology.orchestra.run.smithi135.stdout:(2/159): ceph-fuse-19.0.0-3882.gb7328fed.el9.x8 2.0 MB/s | 890 kB 00:00 2024-06-02T15:25:26.877 INFO:teuthology.orchestra.run.smithi135.stdout:(3/159): ceph-immutable-object-cache-19.0.0-388 1.4 MB/s | 145 kB 00:00 2024-06-02T15:25:26.901 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-06-02T15:25:27.244 INFO:teuthology.orchestra.run.smithi135.stdout:(4/159): ceph-base-19.0.0-3882.gb7328fed.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-06-02T15:25:27.452 INFO:teuthology.orchestra.run.smithi135.stdout:(5/159): ceph-mgr-19.0.0-3882.gb7328fed.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-06-02T15:25:27.553 INFO:teuthology.orchestra.run.smithi135.stdout:(6/159): ceph-mds-19.0.0-3882.gb7328fed.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-06-02T15:25:27.567 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-06-02T15:25:27.936 INFO:teuthology.orchestra.run.smithi135.stdout:(7/159): ceph-mon-19.0.0-3882.gb7328fed.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-06-02T15:25:28.787 INFO:teuthology.orchestra.run.smithi135.stdout:(8/159): ceph-radosgw-19.0.0-3882.gb7328fed.el9 13 MB/s | 11 MB 00:00 2024-06-02T15:25:28.862 INFO:teuthology.orchestra.run.smithi135.stdout:(9/159): ceph-selinux-19.0.0-3882.gb7328fed.el9 336 kB/s | 25 kB 00:00 2024-06-02T15:25:29.459 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.460 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.481 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T15:25:29.482 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.483 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T15:25:29.484 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T15:25:29.487 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T15:25:29.490 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T15:25:29.492 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T15:25:29.567 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-06-02T15:25:29.593 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-06-02T15:25:29.593 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-06-02T15:25:29.593 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-06-02T15:25:29.593 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-06-02T15:25:29.593 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 6.6 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 890 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 145 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 1.5 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 143 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 3.5 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 7.4 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 49 k 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 11 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 49 M 2024-06-02T15:25:29.594 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 273 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 763 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 225 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 27 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 18 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 32 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 747 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 127 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T15:25:29.595 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 161 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 320 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 300 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 100 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 85 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 170 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-06-02T15:25:29.596 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.4 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-06-02T15:25:29.597 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 21 M 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 34 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 2.4 M 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 274 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 4.7 M 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 17 M 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 16 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 25 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T15:25:29.598 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 163 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 501 k 2024-06-02T15:25:29.599 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T15:25:29.600 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 45 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 135 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T15:25:29.601 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T15:25:29.602 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T15:25:29.603 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T15:25:29.604 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T15:25:29.605 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:25:29.606 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 11 Packages 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 227 M 2024-06-02T15:25:29.607 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-06-02T15:25:29.853 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.853 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.865 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T15:25:29.865 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T15:25:29.866 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T15:25:29.867 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T15:25:29.869 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T15:25:29.871 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T15:25:29.872 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T15:25:29.935 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 6.6 k 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 890 k 2024-06-02T15:25:29.952 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 145 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 1.5 M 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 143 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 3.5 M 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 7.4 M 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 49 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 11 M 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 49 M 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 273 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 763 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 225 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 27 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 18 k 2024-06-02T15:25:29.953 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 32 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 747 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 127 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 161 k 2024-06-02T15:25:29.954 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 320 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 300 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 100 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 85 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 170 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T15:25:29.955 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.4 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 3.1 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 21 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 34 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 2.4 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 274 k 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 4.7 M 2024-06-02T15:25:29.956 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 17 M 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3882.gb7328fed.el9 ceph-noarch 16 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 25 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 163 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T15:25:29.957 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 501 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 5.4 M 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T15:25:29.958 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 45 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.0.0-3882.gb7328fed.el9 ceph 135 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T15:25:29.959 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T15:25:29.960 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T15:25:29.961 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T15:25:29.962 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T15:25:29.963 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T15:25:29.965 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T15:25:29.965 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T15:25:29.965 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T15:25:29.965 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-06-02T15:25:29.966 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-06-02T15:25:29.967 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-06-02T15:25:29.967 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 11 Packages 2024-06-02T15:25:29.967 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:25:29.967 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 227 M 2024-06-02T15:25:29.967 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-06-02T15:25:30.762 INFO:teuthology.orchestra.run.smithi090.stdout:(1/159): ceph-19.0.0-3882.gb7328fed.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-06-02T15:25:30.794 INFO:teuthology.orchestra.run.smithi120.stdout:(1/159): ceph-19.0.0-3882.gb7328fed.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-06-02T15:25:31.921 INFO:teuthology.orchestra.run.smithi090.stdout:(2/159): ceph-fuse-19.0.0-3882.gb7328fed.el9.x8 769 kB/s | 890 kB 00:01 2024-06-02T15:25:32.063 INFO:teuthology.orchestra.run.smithi090.stdout:(3/159): ceph-immutable-object-cache-19.0.0-388 1.0 MB/s | 145 kB 00:00 2024-06-02T15:25:32.446 INFO:teuthology.orchestra.run.smithi135.stdout:(10/159): ceph-test-19.0.0-3882.gb7328fed.el9.x 14 MB/s | 49 MB 00:03 2024-06-02T15:25:32.511 INFO:teuthology.orchestra.run.smithi120.stdout:(2/159): ceph-fuse-19.0.0-3882.gb7328fed.el9.x8 519 kB/s | 890 kB 00:01 2024-06-02T15:25:32.546 INFO:teuthology.orchestra.run.smithi135.stdout:(11/159): cephfs-mirror-19.0.0-3882.gb7328fed.e 2.2 MB/s | 225 kB 00:00 2024-06-02T15:25:32.638 INFO:teuthology.orchestra.run.smithi135.stdout:(12/159): cephfs-shell-19.0.0-3882.gb7328fed.el 295 kB/s | 27 kB 00:00 2024-06-02T15:25:32.730 INFO:teuthology.orchestra.run.smithi135.stdout:(13/159): libcephfs-devel-19.0.0-3882.gb7328fed 346 kB/s | 32 kB 00:00 2024-06-02T15:25:32.760 INFO:teuthology.orchestra.run.smithi120.stdout:(3/159): ceph-immutable-object-cache-19.0.0-388 582 kB/s | 145 kB 00:00 2024-06-02T15:25:32.939 INFO:teuthology.orchestra.run.smithi135.stdout:(14/159): libcephfs2-19.0.0-3882.gb7328fed.el9. 3.5 MB/s | 747 kB 00:00 2024-06-02T15:25:33.039 INFO:teuthology.orchestra.run.smithi090.stdout:(4/159): ceph-mds-19.0.0-3882.gb7328fed.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-06-02T15:25:33.047 INFO:teuthology.orchestra.run.smithi135.stdout:(15/159): libcephsqlite-19.0.0-3882.gb7328fed.e 1.5 MB/s | 163 kB 00:00 2024-06-02T15:25:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:(5/159): ceph-base-19.0.0-3882.gb7328fed.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-06-02T15:25:33.290 INFO:teuthology.orchestra.run.smithi135.stdout:(16/159): ceph-common-19.0.0-3882.gb7328fed.el9 2.9 MB/s | 21 MB 00:07 2024-06-02T15:25:33.315 INFO:teuthology.orchestra.run.smithi135.stdout:(17/159): librados-devel-19.0.0-3882.gb7328fed. 474 kB/s | 127 kB 00:00 2024-06-02T15:25:33.590 INFO:teuthology.orchestra.run.smithi135.stdout:(18/159): libradosstriper1-19.0.0-3882.gb7328fe 1.6 MB/s | 501 kB 00:00 2024-06-02T15:25:33.681 INFO:teuthology.orchestra.run.smithi090.stdout:(6/159): ceph-mgr-19.0.0-3882.gb7328fed.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-06-02T15:25:33.715 INFO:teuthology.orchestra.run.smithi135.stdout:(19/159): python3-ceph-argparse-19.0.0-3882.gb7 361 kB/s | 45 kB 00:00 2024-06-02T15:25:33.807 INFO:teuthology.orchestra.run.smithi135.stdout:(20/159): python3-ceph-common-19.0.0-3882.gb732 1.4 MB/s | 135 kB 00:00 2024-06-02T15:25:33.908 INFO:teuthology.orchestra.run.smithi135.stdout:(21/159): python3-cephfs-19.0.0-3882.gb7328fed. 1.6 MB/s | 161 kB 00:00 2024-06-02T15:25:34.066 INFO:teuthology.orchestra.run.smithi135.stdout:(22/159): python3-rados-19.0.0-3882.gb7328fed.e 2.0 MB/s | 320 kB 00:00 2024-06-02T15:25:34.142 INFO:teuthology.orchestra.run.smithi135.stdout:(23/159): librgw2-19.0.0-3882.gb7328fed.el9.x86 6.5 MB/s | 5.4 MB 00:00 2024-06-02T15:25:34.209 INFO:teuthology.orchestra.run.smithi135.stdout:(24/159): python3-rbd-19.0.0-3882.gb7328fed.el9 2.1 MB/s | 300 kB 00:00 2024-06-02T15:25:34.234 INFO:teuthology.orchestra.run.smithi135.stdout:(25/159): python3-rgw-19.0.0-3882.gb7328fed.el9 1.1 MB/s | 100 kB 00:00 2024-06-02T15:25:34.268 INFO:teuthology.orchestra.run.smithi135.stdout:(26/159): rbd-fuse-19.0.0-3882.gb7328fed.el9.x8 1.4 MB/s | 85 kB 00:00 2024-06-02T15:25:34.340 INFO:teuthology.orchestra.run.smithi090.stdout:(7/159): ceph-mon-19.0.0-3882.gb7328fed.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-06-02T15:25:34.343 INFO:teuthology.orchestra.run.smithi135.stdout:(27/159): rbd-nbd-19.0.0-3882.gb7328fed.el9.x86 2.2 MB/s | 170 kB 00:00 2024-06-02T15:25:34.419 INFO:teuthology.orchestra.run.smithi135.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3882.g 456 kB/s | 34 kB 00:00 2024-06-02T15:25:34.502 INFO:teuthology.orchestra.run.smithi135.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3882.gb7328fe 1.7 MB/s | 143 kB 00:00 2024-06-02T15:25:34.628 INFO:teuthology.orchestra.run.smithi135.stdout:(30/159): rbd-mirror-19.0.0-3882.gb7328fed.el9. 7.9 MB/s | 3.1 MB 00:00 2024-06-02T15:25:35.528 INFO:teuthology.orchestra.run.smithi135.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-06-02T15:25:35.779 INFO:teuthology.orchestra.run.smithi135.stdout:(32/159): ceph-osd-19.0.0-3882.gb7328fed.el9.x8 2.0 MB/s | 17 MB 00:08 2024-06-02T15:25:35.837 INFO:teuthology.orchestra.run.smithi135.stdout:(33/159): ceph-mgr-dashboard-19.0.0-3882.gb7328 2.7 MB/s | 3.5 MB 00:01 2024-06-02T15:25:35.863 INFO:teuthology.orchestra.run.smithi135.stdout:(34/159): ceph-mgr-modules-core-19.0.0-3882.gb7 822 kB/s | 274 kB 00:00 2024-06-02T15:25:35.880 INFO:teuthology.orchestra.run.smithi135.stdout:(35/159): ceph-mgr-rook-19.0.0-3882.gb7328fed.e 490 kB/s | 49 kB 00:00 2024-06-02T15:25:35.897 INFO:teuthology.orchestra.run.smithi135.stdout:(36/159): ceph-prometheus-alerts-19.0.0-3882.gb 280 kB/s | 16 kB 00:00 2024-06-02T15:25:35.947 INFO:teuthology.orchestra.run.smithi135.stdout:(37/159): ceph-volume-19.0.0-3882.gb7328fed.el9 3.2 MB/s | 273 kB 00:00 2024-06-02T15:25:35.964 INFO:teuthology.orchestra.run.smithi135.stdout:(38/159): cephfs-top-19.0.0-3882.gb7328fed.el9. 271 kB/s | 18 kB 00:00 2024-06-02T15:25:36.028 INFO:teuthology.orchestra.run.smithi120.stdout:(4/159): ceph-mds-19.0.0-3882.gb7328fed.el9.x86 743 kB/s | 2.4 MB 00:03 2024-06-02T15:25:36.131 INFO:teuthology.orchestra.run.smithi135.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-06-02T15:25:36.173 INFO:teuthology.orchestra.run.smithi135.stdout:(40/159): cephadm-19.0.0-3882.gb7328fed.el9.noa 2.5 MB/s | 763 kB 00:00 2024-06-02T15:25:36.199 INFO:teuthology.orchestra.run.smithi135.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-06-02T15:25:36.241 INFO:teuthology.orchestra.run.smithi135.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 195 kB 00:00 2024-06-02T15:25:36.299 INFO:teuthology.orchestra.run.smithi135.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-02T15:25:36.333 INFO:teuthology.orchestra.run.smithi135.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-06-02T15:25:36.358 INFO:teuthology.orchestra.run.smithi135.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-02T15:25:36.374 INFO:teuthology.orchestra.run.smithi090.stdout:(8/159): ceph-radosgw-19.0.0-3882.gb7328fed.el9 5.3 MB/s | 11 MB 00:02 2024-06-02T15:25:36.392 INFO:teuthology.orchestra.run.smithi135.stdout:(46/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-06-02T15:25:36.417 INFO:teuthology.orchestra.run.smithi135.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 1.4 MB/s | 125 kB 00:00 2024-06-02T15:25:36.441 INFO:teuthology.orchestra.run.smithi090.stdout:(9/159): ceph-selinux-19.0.0-3882.gb7328fed.el9 382 kB/s | 25 kB 00:00 2024-06-02T15:25:36.442 INFO:teuthology.orchestra.run.smithi135.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-06-02T15:25:36.678 INFO:teuthology.orchestra.run.smithi120.stdout:(5/159): ceph-base-19.0.0-3882.gb7328fed.el9.x8 884 kB/s | 5.4 MB 00:06 2024-06-02T15:25:36.860 INFO:teuthology.orchestra.run.smithi135.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 256 kB/s | 107 kB 00:00 2024-06-02T15:25:37.118 INFO:teuthology.orchestra.run.smithi135.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 84 kB/s | 22 kB 00:00 2024-06-02T15:25:37.162 INFO:teuthology.orchestra.run.smithi120.stdout:(6/159): ceph-common-19.0.0-3882.gb7328fed.el9. 3.1 MB/s | 21 MB 00:06 2024-06-02T15:25:37.195 INFO:teuthology.orchestra.run.smithi120.stdout:(7/159): ceph-mgr-19.0.0-3882.gb7328fed.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-06-02T15:25:37.618 INFO:teuthology.orchestra.run.smithi090.stdout:(10/159): ceph-osd-19.0.0-3882.gb7328fed.el9.x8 4.3 MB/s | 17 MB 00:03 2024-06-02T15:25:37.741 INFO:teuthology.orchestra.run.smithi090.stdout:(11/159): cephfs-mirror-19.0.0-3882.gb7328fed.e 1.8 MB/s | 225 kB 00:00 2024-06-02T15:25:37.817 INFO:teuthology.orchestra.run.smithi090.stdout:(12/159): cephfs-shell-19.0.0-3882.gb7328fed.el 365 kB/s | 27 kB 00:00 2024-06-02T15:25:38.069 INFO:teuthology.orchestra.run.smithi135.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 571 kB/s | 943 kB 00:01 2024-06-02T15:25:38.094 INFO:teuthology.orchestra.run.smithi135.stdout:(52/159): automake-1.16.2-8.el9.noarch.rpm 405 kB/s | 689 kB 00:01 2024-06-02T15:25:38.109 INFO:teuthology.orchestra.run.smithi090.stdout:(13/159): ceph-common-19.0.0-3882.gb7328fed.el9 2.7 MB/s | 21 MB 00:07 2024-06-02T15:25:38.119 INFO:teuthology.orchestra.run.smithi135.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 313 kB/s | 313 kB 00:01 2024-06-02T15:25:38.134 INFO:teuthology.orchestra.run.smithi090.stdout:(14/159): libcephfs-devel-19.0.0-3882.gb7328fed 100 kB/s | 32 kB 00:00 2024-06-02T15:25:38.243 INFO:teuthology.orchestra.run.smithi090.stdout:(15/159): libcephsqlite-19.0.0-3882.gb7328fed.e 1.5 MB/s | 163 kB 00:00 2024-06-02T15:25:38.328 INFO:teuthology.orchestra.run.smithi135.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 126 kB/s | 33 kB 00:00 2024-06-02T15:25:38.335 INFO:teuthology.orchestra.run.smithi090.stdout:(16/159): libcephfs2-19.0.0-3882.gb7328fed.el9. 3.3 MB/s | 747 kB 00:00 2024-06-02T15:25:38.368 INFO:teuthology.orchestra.run.smithi090.stdout:(17/159): librados-devel-19.0.0-3882.gb7328fed. 1.0 MB/s | 127 kB 00:00 2024-06-02T15:25:38.378 INFO:teuthology.orchestra.run.smithi135.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 70 kB/s | 18 kB 00:00 2024-06-02T15:25:38.461 INFO:teuthology.orchestra.run.smithi090.stdout:(18/159): libradosstriper1-19.0.0-3882.gb7328fe 3.9 MB/s | 501 kB 00:00 2024-06-02T15:25:38.504 INFO:teuthology.orchestra.run.smithi135.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 63 kB/s | 11 kB 00:00 2024-06-02T15:25:38.519 INFO:teuthology.orchestra.run.smithi090.stdout:(19/159): python3-ceph-argparse-19.0.0-3882.gb7 782 kB/s | 45 kB 00:00 2024-06-02T15:25:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:(20/159): python3-ceph-common-19.0.0-3882.gb732 2.0 MB/s | 135 kB 00:00 2024-06-02T15:25:38.654 INFO:teuthology.orchestra.run.smithi090.stdout:(21/159): python3-cephfs-19.0.0-3882.gb7328fed. 2.4 MB/s | 161 kB 00:00 2024-06-02T15:25:38.762 INFO:teuthology.orchestra.run.smithi090.stdout:(22/159): python3-rados-19.0.0-3882.gb7328fed.e 2.9 MB/s | 320 kB 00:00 2024-06-02T15:25:38.838 INFO:teuthology.orchestra.run.smithi135.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 136 kB/s | 45 kB 00:00 2024-06-02T15:25:38.863 INFO:teuthology.orchestra.run.smithi090.stdout:(23/159): python3-rbd-19.0.0-3882.gb7328fed.el9 3.0 MB/s | 300 kB 00:00 2024-06-02T15:25:38.921 INFO:teuthology.orchestra.run.smithi090.stdout:(24/159): python3-rgw-19.0.0-3882.gb7328fed.el9 1.7 MB/s | 100 kB 00:00 2024-06-02T15:25:38.938 INFO:teuthology.orchestra.run.smithi135.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 286 kB/s | 160 kB 00:00 2024-06-02T15:25:38.972 INFO:teuthology.orchestra.run.smithi090.stdout:(25/159): rbd-fuse-19.0.0-3882.gb7328fed.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-02T15:25:39.112 INFO:teuthology.orchestra.run.smithi120.stdout:(8/159): ceph-mon-19.0.0-3882.gb7328fed.el9.x86 1.9 MB/s | 4.7 MB 00:02 2024-06-02T15:25:39.212 INFO:teuthology.orchestra.run.smithi120.stdout:(9/159): ceph-selinux-19.0.0-3882.gb7328fed.el9 252 kB/s | 25 kB 00:00 2024-06-02T15:25:39.457 INFO:teuthology.orchestra.run.smithi090.stdout:(26/159): librgw2-19.0.0-3882.gb7328fed.el9.x86 5.0 MB/s | 5.4 MB 00:01 2024-06-02T15:25:39.513 INFO:teuthology.orchestra.run.smithi135.stdout:(59/159): libtool-2.4.6-45.el9.x86_64.rpm 1.0 MB/s | 582 kB 00:00 2024-06-02T15:25:39.524 INFO:teuthology.orchestra.run.smithi090.stdout:(27/159): rbd-nbd-19.0.0-3882.gb7328fed.el9.x86 2.5 MB/s | 170 kB 00:00 2024-06-02T15:25:39.563 INFO:teuthology.orchestra.run.smithi120.stdout:(10/159): ceph-osd-19.0.0-3882.gb7328fed.el9.x8 7.0 MB/s | 17 MB 00:02 2024-06-02T15:25:39.613 INFO:teuthology.orchestra.run.smithi120.stdout:(11/159): cephfs-mirror-19.0.0-3882.gb7328fed.e 4.4 MB/s | 225 kB 00:00 2024-06-02T15:25:39.616 INFO:teuthology.orchestra.run.smithi090.stdout:(28/159): rbd-mirror-19.0.0-3882.gb7328fed.el9. 4.8 MB/s | 3.1 MB 00:00 2024-06-02T15:25:39.641 INFO:teuthology.orchestra.run.smithi090.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3882.g 294 kB/s | 34 kB 00:00 2024-06-02T15:25:39.655 INFO:teuthology.orchestra.run.smithi120.stdout:(12/159): cephfs-shell-19.0.0-3882.gb7328fed.el 651 kB/s | 27 kB 00:00 2024-06-02T15:25:39.675 INFO:teuthology.orchestra.run.smithi090.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3882.gb7328fe 2.4 MB/s | 143 kB 00:00 2024-06-02T15:25:39.697 INFO:teuthology.orchestra.run.smithi120.stdout:(13/159): libcephfs-devel-19.0.0-3882.gb7328fed 765 kB/s | 32 kB 00:00 2024-06-02T15:25:39.772 INFO:teuthology.orchestra.run.smithi135.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 98 kB/s | 25 kB 00:00 2024-06-02T15:25:39.872 INFO:teuthology.orchestra.run.smithi135.stdout:(61/159): librdkafka-1.6.1-102.el9.x86_64.rpm 640 kB/s | 662 kB 00:01 2024-06-02T15:25:39.880 INFO:teuthology.orchestra.run.smithi120.stdout:(14/159): libcephfs2-19.0.0-3882.gb7328fed.el9. 4.0 MB/s | 747 kB 00:00 2024-06-02T15:25:39.939 INFO:teuthology.orchestra.run.smithi120.stdout:(15/159): libcephsqlite-19.0.0-3882.gb7328fed.e 2.7 MB/s | 163 kB 00:00 2024-06-02T15:25:39.998 INFO:teuthology.orchestra.run.smithi120.stdout:(16/159): librados-devel-19.0.0-3882.gb7328fed. 2.1 MB/s | 127 kB 00:00 2024-06-02T15:25:40.131 INFO:teuthology.orchestra.run.smithi135.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 132 kB/s | 47 kB 00:00 2024-06-02T15:25:40.132 INFO:teuthology.orchestra.run.smithi120.stdout:(17/159): libradosstriper1-19.0.0-3882.gb7328fe 3.7 MB/s | 501 kB 00:00 2024-06-02T15:25:40.367 INFO:teuthology.orchestra.run.smithi090.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3882.gb7328 4.9 MB/s | 3.5 MB 00:00 2024-06-02T15:25:40.459 INFO:teuthology.orchestra.run.smithi090.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3882.gb7 3.0 MB/s | 274 kB 00:00 2024-06-02T15:25:40.551 INFO:teuthology.orchestra.run.smithi090.stdout:(33/159): ceph-mgr-rook-19.0.0-3882.gb7328fed.e 544 kB/s | 49 kB 00:00 2024-06-02T15:25:40.643 INFO:teuthology.orchestra.run.smithi090.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3882.gb 181 kB/s | 16 kB 00:00 2024-06-02T15:25:40.743 INFO:teuthology.orchestra.run.smithi090.stdout:(35/159): ceph-volume-19.0.0-3882.gb7328fed.el9 2.7 MB/s | 273 kB 00:00 2024-06-02T15:25:40.756 INFO:teuthology.orchestra.run.smithi135.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 331 kB/s | 292 kB 00:00 2024-06-02T15:25:40.848 INFO:teuthology.orchestra.run.smithi135.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 263 kB/s | 188 kB 00:00 2024-06-02T15:25:40.915 INFO:teuthology.orchestra.run.smithi135.stdout:(65/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.1 MB/s | 3.0 MB 00:02 2024-06-02T15:25:40.969 INFO:teuthology.orchestra.run.smithi090.stdout:(36/159): cephadm-19.0.0-3882.gb7328fed.el9.noa 3.3 MB/s | 763 kB 00:00 2024-06-02T15:25:41.024 INFO:teuthology.orchestra.run.smithi135.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 132 kB/s | 35 kB 00:00 2024-06-02T15:25:41.061 INFO:teuthology.orchestra.run.smithi090.stdout:(37/159): cephfs-top-19.0.0-3882.gb7328fed.el9. 201 kB/s | 18 kB 00:00 2024-06-02T15:25:41.157 INFO:teuthology.orchestra.run.smithi120.stdout:(18/159): librgw2-19.0.0-3882.gb7328fed.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-06-02T15:25:41.190 INFO:teuthology.orchestra.run.smithi120.stdout:(19/159): python3-ceph-argparse-19.0.0-3882.gb7 1.3 MB/s | 45 kB 00:00 2024-06-02T15:25:41.241 INFO:teuthology.orchestra.run.smithi120.stdout:(20/159): python3-ceph-common-19.0.0-3882.gb732 2.7 MB/s | 135 kB 00:00 2024-06-02T15:25:41.278 INFO:teuthology.orchestra.run.smithi090.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-06-02T15:25:41.291 INFO:teuthology.orchestra.run.smithi120.stdout:(21/159): python3-cephfs-19.0.0-3882.gb7328fed. 3.2 MB/s | 161 kB 00:00 2024-06-02T15:25:41.303 INFO:teuthology.orchestra.run.smithi090.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 333 kB/s | 80 kB 00:00 2024-06-02T15:25:41.358 INFO:teuthology.orchestra.run.smithi120.stdout:(22/159): python3-rados-19.0.0-3882.gb7328fed.e 4.7 MB/s | 320 kB 00:00 2024-06-02T15:25:41.420 INFO:teuthology.orchestra.run.smithi090.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-06-02T15:25:41.425 INFO:teuthology.orchestra.run.smithi120.stdout:(23/159): python3-rbd-19.0.0-3882.gb7328fed.el9 4.4 MB/s | 300 kB 00:00 2024-06-02T15:25:41.484 INFO:teuthology.orchestra.run.smithi120.stdout:(24/159): python3-rgw-19.0.0-3882.gb7328fed.el9 1.7 MB/s | 100 kB 00:00 2024-06-02T15:25:41.512 INFO:teuthology.orchestra.run.smithi090.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-06-02T15:25:41.526 INFO:teuthology.orchestra.run.smithi120.stdout:(25/159): rbd-fuse-19.0.0-3882.gb7328fed.el9.x8 2.0 MB/s | 85 kB 00:00 2024-06-02T15:25:41.563 INFO:teuthology.orchestra.run.smithi090.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-06-02T15:25:41.638 INFO:teuthology.orchestra.run.smithi090.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-06-02T15:25:41.672 INFO:teuthology.orchestra.run.smithi090.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 987 kB/s | 106 kB 00:00 2024-06-02T15:25:41.697 INFO:teuthology.orchestra.run.smithi090.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-02T15:25:41.730 INFO:teuthology.orchestra.run.smithi090.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-02T15:25:41.764 INFO:teuthology.orchestra.run.smithi090.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-06-02T15:25:41.826 INFO:teuthology.orchestra.run.smithi120.stdout:(26/159): ceph-radosgw-19.0.0-3882.gb7328fed.el 2.3 MB/s | 11 MB 00:04 2024-06-02T15:25:41.926 INFO:teuthology.orchestra.run.smithi120.stdout:(27/159): rbd-mirror-19.0.0-3882.gb7328fed.el9. 7.7 MB/s | 3.1 MB 00:00 2024-06-02T15:25:41.933 INFO:teuthology.orchestra.run.smithi090.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-06-02T15:25:41.952 INFO:teuthology.orchestra.run.smithi120.stdout:(28/159): rbd-nbd-19.0.0-3882.gb7328fed.el9.x86 1.3 MB/s | 170 kB 00:00 2024-06-02T15:25:41.968 INFO:teuthology.orchestra.run.smithi120.stdout:(29/159): ceph-grafana-dashboards-19.0.0-3882.g 817 kB/s | 34 kB 00:00 2024-06-02T15:25:42.000 INFO:teuthology.orchestra.run.smithi090.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-06-02T15:25:42.008 INFO:teuthology.orchestra.run.smithi135.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-06-02T15:25:42.019 INFO:teuthology.orchestra.run.smithi120.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3882.gb7328fe 2.1 MB/s | 143 kB 00:00 2024-06-02T15:25:42.033 INFO:teuthology.orchestra.run.smithi090.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-06-02T15:25:42.059 INFO:teuthology.orchestra.run.smithi090.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 375 kB/s | 22 kB 00:00 2024-06-02T15:25:42.101 INFO:teuthology.orchestra.run.smithi090.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-06-02T15:25:42.126 INFO:teuthology.orchestra.run.smithi090.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-06-02T15:25:42.168 INFO:teuthology.orchestra.run.smithi090.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 446 kB/s | 18 kB 00:00 2024-06-02T15:25:42.210 INFO:teuthology.orchestra.run.smithi090.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 273 kB/s | 11 kB 00:00 2024-06-02T15:25:42.269 INFO:teuthology.orchestra.run.smithi090.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-06-02T15:25:42.361 INFO:teuthology.orchestra.run.smithi090.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-06-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi120.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3882.gb7328 9.0 MB/s | 3.5 MB 00:00 2024-06-02T15:25:42.386 INFO:teuthology.orchestra.run.smithi090.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-06-02T15:25:42.408 INFO:teuthology.orchestra.run.smithi135.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-06-02T15:25:42.420 INFO:teuthology.orchestra.run.smithi120.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3882.gb7 4.6 MB/s | 274 kB 00:00 2024-06-02T15:25:42.470 INFO:teuthology.orchestra.run.smithi120.stdout:(33/159): ceph-mgr-rook-19.0.0-3882.gb7328fed.e 986 kB/s | 49 kB 00:00 2024-06-02T15:25:42.512 INFO:teuthology.orchestra.run.smithi120.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3882.gb 393 kB/s | 16 kB 00:00 2024-06-02T15:25:42.562 INFO:teuthology.orchestra.run.smithi120.stdout:(35/159): ceph-volume-19.0.0-3882.gb7328fed.el9 5.3 MB/s | 273 kB 00:00 2024-06-02T15:25:42.595 INFO:teuthology.orchestra.run.smithi090.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-06-02T15:25:42.637 INFO:teuthology.orchestra.run.smithi090.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-06-02T15:25:42.662 INFO:teuthology.orchestra.run.smithi090.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-06-02T15:25:42.663 INFO:teuthology.orchestra.run.smithi120.stdout:(36/159): cephadm-19.0.0-3882.gb7328fed.el9.noa 7.5 MB/s | 763 kB 00:00 2024-06-02T15:25:42.688 INFO:teuthology.orchestra.run.smithi090.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 958 kB/s | 47 kB 00:00 2024-06-02T15:25:42.705 INFO:teuthology.orchestra.run.smithi120.stdout:(37/159): cephfs-top-19.0.0-3882.gb7328fed.el9. 439 kB/s | 18 kB 00:00 2024-06-02T15:25:42.922 INFO:teuthology.orchestra.run.smithi120.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-06-02T15:25:42.975 INFO:teuthology.orchestra.run.smithi135.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 235 kB/s | 133 kB 00:00 2024-06-02T15:25:43.072 INFO:teuthology.orchestra.run.smithi120.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-06-02T15:25:43.131 INFO:teuthology.orchestra.run.smithi120.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-06-02T15:25:43.190 INFO:teuthology.orchestra.run.smithi120.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-06-02T15:25:43.290 INFO:teuthology.orchestra.run.smithi120.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-06-02T15:25:43.303 INFO:teuthology.orchestra.run.smithi090.stdout:(63/159): ceph-test-19.0.0-3882.gb7328fed.el9.x 7.2 MB/s | 49 MB 00:06 2024-06-02T15:25:43.338 INFO:teuthology.orchestra.run.smithi090.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 433 kB/s | 292 kB 00:00 2024-06-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi120.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-02T15:25:43.364 INFO:teuthology.orchestra.run.smithi090.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 279 kB/s | 188 kB 00:00 2024-06-02T15:25:43.407 INFO:teuthology.orchestra.run.smithi120.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-02T15:25:43.431 INFO:teuthology.orchestra.run.smithi090.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 278 kB/s | 35 kB 00:00 2024-06-02T15:25:43.466 INFO:teuthology.orchestra.run.smithi120.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-06-02T15:25:43.525 INFO:teuthology.orchestra.run.smithi120.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-06-02T15:25:43.598 INFO:teuthology.orchestra.run.smithi090.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-02T15:25:43.690 INFO:teuthology.orchestra.run.smithi090.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-06-02T15:25:43.691 INFO:teuthology.orchestra.run.smithi120.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-06-02T15:25:43.757 INFO:teuthology.orchestra.run.smithi090.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-06-02T15:25:43.767 INFO:teuthology.orchestra.run.smithi120.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-06-02T15:25:43.782 INFO:teuthology.orchestra.run.smithi090.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-06-02T15:25:43.809 INFO:teuthology.orchestra.run.smithi120.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-02T15:25:43.834 INFO:teuthology.orchestra.run.smithi135.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 499 kB/s | 911 kB 00:01 2024-06-02T15:25:43.840 INFO:teuthology.orchestra.run.smithi090.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-06-02T15:25:43.842 INFO:teuthology.orchestra.run.smithi120.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 651 kB/s | 22 kB 00:00 2024-06-02T15:25:43.893 INFO:teuthology.orchestra.run.smithi120.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-02T15:25:43.909 INFO:teuthology.orchestra.run.smithi090.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-06-02T15:25:43.926 INFO:teuthology.orchestra.run.smithi120.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 983 kB/s | 33 kB 00:00 2024-06-02T15:25:43.959 INFO:teuthology.orchestra.run.smithi090.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 741 kB/s | 87 kB 00:00 2024-06-02T15:25:44.010 INFO:teuthology.orchestra.run.smithi090.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-06-02T15:25:44.027 INFO:teuthology.orchestra.run.smithi120.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-06-02T15:25:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-06-02T15:25:44.072 INFO:teuthology.orchestra.run.smithi120.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 408 kB/s | 18 kB 00:00 2024-06-02T15:25:44.120 INFO:teuthology.orchestra.run.smithi120.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 237 kB/s | 11 kB 00:00 2024-06-02T15:25:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-06-02T15:25:44.162 INFO:teuthology.orchestra.run.smithi120.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-06-02T15:25:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 520 kB/s | 26 kB 00:00 2024-06-02T15:25:44.203 INFO:teuthology.orchestra.run.smithi120.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-02T15:25:44.288 INFO:teuthology.orchestra.run.smithi120.stdout:(58/159): ceph-mgr-diskprediction-local-19.0.0- 3.3 MB/s | 7.4 MB 00:02 2024-06-02T15:25:44.302 INFO:teuthology.orchestra.run.smithi090.stdout:(78/159): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-06-02T15:25:44.321 INFO:teuthology.orchestra.run.smithi120.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-06-02T15:25:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 128 kB/s | 19 kB 00:00 2024-06-02T15:25:44.362 INFO:teuthology.orchestra.run.smithi090.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-02T15:25:44.371 INFO:teuthology.orchestra.run.smithi120.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-06-02T15:25:44.395 INFO:teuthology.orchestra.run.smithi090.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 523 kB/s | 35 kB 00:00 2024-06-02T15:25:44.413 INFO:teuthology.orchestra.run.smithi120.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-02T15:25:44.472 INFO:teuthology.orchestra.run.smithi120.stdout:(62/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-06-02T15:25:44.497 INFO:teuthology.orchestra.run.smithi120.stdout:(63/159): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-06-02T15:25:44.523 INFO:teuthology.orchestra.run.smithi120.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-02T15:25:44.538 INFO:teuthology.orchestra.run.smithi090.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.5 MB/s | 6.0 MB 00:00 2024-06-02T15:25:44.548 INFO:teuthology.orchestra.run.smithi120.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-02T15:25:44.571 INFO:teuthology.orchestra.run.smithi090.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-06-02T15:25:44.605 INFO:teuthology.orchestra.run.smithi090.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-06-02T15:25:44.648 INFO:teuthology.orchestra.run.smithi120.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-06-02T15:25:44.672 INFO:teuthology.orchestra.run.smithi090.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 772 kB/s | 77 kB 00:00 2024-06-02T15:25:44.697 INFO:teuthology.orchestra.run.smithi090.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 463 kB/s | 42 kB 00:00 2024-06-02T15:25:44.715 INFO:teuthology.orchestra.run.smithi120.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-06-02T15:25:44.739 INFO:teuthology.orchestra.run.smithi090.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-06-02T15:25:44.767 INFO:teuthology.orchestra.run.smithi135.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-06-02T15:25:44.781 INFO:teuthology.orchestra.run.smithi090.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-06-02T15:25:44.782 INFO:teuthology.orchestra.run.smithi120.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-02T15:25:44.815 INFO:teuthology.orchestra.run.smithi090.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-06-02T15:25:44.824 INFO:teuthology.orchestra.run.smithi120.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-06-02T15:25:44.840 INFO:teuthology.orchestra.run.smithi090.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 882 kB/s | 51 kB 00:00 2024-06-02T15:25:44.849 INFO:teuthology.orchestra.run.smithi120.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-06-02T15:25:44.865 INFO:teuthology.orchestra.run.smithi090.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-06-02T15:25:44.908 INFO:teuthology.orchestra.run.smithi090.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-02T15:25:44.909 INFO:teuthology.orchestra.run.smithi120.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-02T15:25:44.950 INFO:teuthology.orchestra.run.smithi090.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-06-02T15:25:44.983 INFO:teuthology.orchestra.run.smithi120.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-06-02T15:25:45.009 INFO:teuthology.orchestra.run.smithi120.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-06-02T15:25:45.075 INFO:teuthology.orchestra.run.smithi090.stdout:(94/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 5.1 MB 00:00 2024-06-02T15:25:45.109 INFO:teuthology.orchestra.run.smithi090.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-06-02T15:25:45.167 INFO:teuthology.orchestra.run.smithi120.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 387 kB/s | 61 kB 00:00 2024-06-02T15:25:45.176 INFO:teuthology.orchestra.run.smithi135.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 214 kB/s | 87 kB 00:00 2024-06-02T15:25:45.177 INFO:teuthology.orchestra.run.smithi090.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 356 kB/s | 36 kB 00:00 2024-06-02T15:25:45.202 INFO:teuthology.orchestra.run.smithi090.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 513 kB/s | 47 kB 00:00 2024-06-02T15:25:45.218 INFO:teuthology.orchestra.run.smithi135.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 747 kB/s | 1.0 MB 00:01 2024-06-02T15:25:45.251 INFO:teuthology.orchestra.run.smithi120.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-06-02T15:25:45.252 INFO:teuthology.orchestra.run.smithi090.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2024-06-02T15:25:45.276 INFO:teuthology.orchestra.run.smithi120.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-02T15:25:45.294 INFO:teuthology.orchestra.run.smithi090.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-06-02T15:25:45.302 INFO:teuthology.orchestra.run.smithi120.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-02T15:25:45.319 INFO:teuthology.orchestra.run.smithi120.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-06-02T15:25:45.320 INFO:teuthology.orchestra.run.smithi090.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-06-02T15:25:45.336 INFO:teuthology.orchestra.run.smithi120.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-06-02T15:25:45.370 INFO:teuthology.orchestra.run.smithi120.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-02T15:25:45.371 INFO:teuthology.orchestra.run.smithi090.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 855 kB/s | 64 kB 00:00 2024-06-02T15:25:45.387 INFO:teuthology.orchestra.run.smithi120.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 686 kB/s | 35 kB 00:00 2024-06-02T15:25:45.395 INFO:teuthology.orchestra.run.smithi090.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-06-02T15:25:45.437 INFO:teuthology.orchestra.run.smithi120.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-06-02T15:25:45.463 INFO:teuthology.orchestra.run.smithi090.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-06-02T15:25:45.529 INFO:teuthology.orchestra.run.smithi120.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-06-02T15:25:45.554 INFO:teuthology.orchestra.run.smithi120.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-06-02T15:25:45.555 INFO:teuthology.orchestra.run.smithi090.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-06-02T15:25:45.568 INFO:teuthology.orchestra.run.smithi135.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 175 kB/s | 61 kB 00:00 2024-06-02T15:25:45.579 INFO:teuthology.orchestra.run.smithi120.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-02T15:25:45.597 INFO:teuthology.orchestra.run.smithi120.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-06-02T15:25:45.622 INFO:teuthology.orchestra.run.smithi120.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-06-02T15:25:45.647 INFO:teuthology.orchestra.run.smithi120.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-02T15:25:45.672 INFO:teuthology.orchestra.run.smithi120.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-02T15:25:45.698 INFO:teuthology.orchestra.run.smithi120.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-02T15:25:45.723 INFO:teuthology.orchestra.run.smithi120.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-02T15:25:45.774 INFO:teuthology.orchestra.run.smithi120.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-06-02T15:25:45.816 INFO:teuthology.orchestra.run.smithi120.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-02T15:25:45.855 INFO:teuthology.orchestra.run.smithi090.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-06-02T15:25:45.866 INFO:teuthology.orchestra.run.smithi120.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-02T15:25:45.908 INFO:teuthology.orchestra.run.smithi120.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-06-02T15:25:46.060 INFO:teuthology.orchestra.run.smithi135.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 496 kB/s | 244 kB 00:00 2024-06-02T15:25:46.066 INFO:teuthology.orchestra.run.smithi120.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-06-02T15:25:46.092 INFO:teuthology.orchestra.run.smithi120.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-06-02T15:25:46.200 INFO:teuthology.orchestra.run.smithi120.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-06-02T15:25:46.226 INFO:teuthology.orchestra.run.smithi120.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-06-02T15:25:46.251 INFO:teuthology.orchestra.run.smithi120.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-02T15:25:46.276 INFO:teuthology.orchestra.run.smithi120.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-02T15:25:46.302 INFO:teuthology.orchestra.run.smithi120.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-06-02T15:25:46.319 INFO:teuthology.orchestra.run.smithi120.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 534 kB/s | 22 kB 00:00 2024-06-02T15:25:46.352 INFO:teuthology.orchestra.run.smithi120.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 17 MB/s | 863 kB 00:00 2024-06-02T15:25:46.389 INFO:teuthology.orchestra.run.smithi090.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 593 kB/s | 548 kB 00:00 2024-06-02T15:25:46.414 INFO:teuthology.orchestra.run.smithi090.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 40 kB/s | 19 kB 00:00 2024-06-02T15:25:46.514 INFO:teuthology.orchestra.run.smithi090.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-06-02T15:25:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:(109/159): gperftools-libs-2.9.1-2.el9.x86_64.r 325 kB/s | 309 kB 00:00 2024-06-02T15:25:46.586 INFO:teuthology.orchestra.run.smithi120.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-06-02T15:25:46.612 INFO:teuthology.orchestra.run.smithi120.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-06-02T15:25:46.632 INFO:teuthology.orchestra.run.smithi090.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2024-06-02T15:25:46.645 INFO:teuthology.orchestra.run.smithi120.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-06-02T15:25:46.696 INFO:teuthology.orchestra.run.smithi120.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-06-02T15:25:46.754 INFO:teuthology.orchestra.run.smithi120.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-06-02T15:25:46.796 INFO:teuthology.orchestra.run.smithi090.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-06-02T15:25:46.824 INFO:teuthology.orchestra.run.smithi090.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 261 kB/s | 67 kB 00:00 2024-06-02T15:25:46.829 INFO:teuthology.orchestra.run.smithi120.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 900 kB/s | 67 kB 00:00 2024-06-02T15:25:46.899 INFO:teuthology.orchestra.run.smithi090.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 568 kB/s | 151 kB 00:00 2024-06-02T15:25:46.913 INFO:teuthology.orchestra.run.smithi120.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-06-02T15:25:46.949 INFO:teuthology.orchestra.run.smithi090.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-06-02T15:25:46.980 INFO:teuthology.orchestra.run.smithi120.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-06-02T15:25:46.986 INFO:teuthology.orchestra.run.smithi135.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 268 kB/s | 248 kB 00:00 2024-06-02T15:25:46.992 INFO:teuthology.orchestra.run.smithi090.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-06-02T15:25:47.034 INFO:teuthology.orchestra.run.smithi090.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 387 kB/s | 32 kB 00:00 2024-06-02T15:25:47.039 INFO:teuthology.orchestra.run.smithi120.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-06-02T15:25:47.064 INFO:teuthology.orchestra.run.smithi120.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-06-02T15:25:47.075 INFO:teuthology.orchestra.run.smithi090.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-06-02T15:25:47.081 INFO:teuthology.orchestra.run.smithi120.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-06-02T15:25:47.106 INFO:teuthology.orchestra.run.smithi120.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 772 kB/s | 32 kB 00:00 2024-06-02T15:25:47.109 INFO:teuthology.orchestra.run.smithi090.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-06-02T15:25:47.124 INFO:teuthology.orchestra.run.smithi120.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-06-02T15:25:47.152 INFO:teuthology.orchestra.run.smithi090.stdout:(119/159): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-06-02T15:25:47.157 INFO:teuthology.orchestra.run.smithi120.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-02T15:25:47.195 INFO:teuthology.orchestra.run.smithi090.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-06-02T15:25:47.199 INFO:teuthology.orchestra.run.smithi120.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-06-02T15:25:47.225 INFO:teuthology.orchestra.run.smithi120.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-06-02T15:25:47.253 INFO:teuthology.orchestra.run.smithi135.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 96 kB/s | 26 kB 00:00 2024-06-02T15:25:47.258 INFO:teuthology.orchestra.run.smithi120.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-06-02T15:25:47.261 INFO:teuthology.orchestra.run.smithi090.stdout:(121/159): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-06-02T15:25:47.304 INFO:teuthology.orchestra.run.smithi090.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-06-02T15:25:47.325 INFO:teuthology.orchestra.run.smithi120.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-06-02T15:25:47.351 INFO:teuthology.orchestra.run.smithi120.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-06-02T15:25:47.354 INFO:teuthology.orchestra.run.smithi090.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-06-02T15:25:47.376 INFO:teuthology.orchestra.run.smithi120.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-06-02T15:25:47.388 INFO:teuthology.orchestra.run.smithi090.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-06-02T15:25:47.393 INFO:teuthology.orchestra.run.smithi120.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 420 kB/s | 18 kB 00:00 2024-06-02T15:25:47.418 INFO:teuthology.orchestra.run.smithi120.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 550 kB/s | 23 kB 00:00 2024-06-02T15:25:47.437 INFO:teuthology.orchestra.run.smithi135.stdout:(78/159): postgresql-server-13.14-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-06-02T15:25:47.443 INFO:teuthology.orchestra.run.smithi120.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-06-02T15:25:47.469 INFO:teuthology.orchestra.run.smithi120.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-06-02T15:25:47.505 INFO:teuthology.orchestra.run.smithi090.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-06-02T15:25:47.528 INFO:teuthology.orchestra.run.smithi120.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-06-02T15:25:47.530 INFO:teuthology.orchestra.run.smithi090.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 101 kB/s | 18 kB 00:00 2024-06-02T15:25:47.556 INFO:teuthology.orchestra.run.smithi090.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 139 kB/s | 23 kB 00:00 2024-06-02T15:25:47.561 INFO:teuthology.orchestra.run.smithi120.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-06-02T15:25:47.581 INFO:teuthology.orchestra.run.smithi090.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-06-02T15:25:47.587 INFO:teuthology.orchestra.run.smithi120.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-06-02T15:25:47.612 INFO:teuthology.orchestra.run.smithi135.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 53 kB/s | 19 kB 00:00 2024-06-02T15:25:47.612 INFO:teuthology.orchestra.run.smithi120.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-02T15:25:47.631 INFO:teuthology.orchestra.run.smithi090.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 265 kB/s | 20 kB 00:00 2024-06-02T15:25:47.646 INFO:teuthology.orchestra.run.smithi120.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-02T15:25:47.656 INFO:teuthology.orchestra.run.smithi090.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 619 kB/s | 46 kB 00:00 2024-06-02T15:25:47.671 INFO:teuthology.orchestra.run.smithi120.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-06-02T15:25:47.696 INFO:teuthology.orchestra.run.smithi120.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-02T15:25:47.722 INFO:teuthology.orchestra.run.smithi120.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 560 kB/s | 28 kB 00:00 2024-06-02T15:25:47.747 INFO:teuthology.orchestra.run.smithi120.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-06-02T15:25:47.748 INFO:teuthology.orchestra.run.smithi090.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-06-02T15:25:47.774 INFO:teuthology.orchestra.run.smithi090.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2024-06-02T15:25:47.781 INFO:teuthology.orchestra.run.smithi120.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-06-02T15:25:47.799 INFO:teuthology.orchestra.run.smithi090.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 408 kB/s | 58 kB 00:00 2024-06-02T15:25:47.806 INFO:teuthology.orchestra.run.smithi120.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-06-02T15:25:47.831 INFO:teuthology.orchestra.run.smithi120.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-06-02T15:25:47.841 INFO:teuthology.orchestra.run.smithi090.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 248 kB/s | 16 kB 00:00 2024-06-02T15:25:47.857 INFO:teuthology.orchestra.run.smithi120.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-02T15:25:47.875 INFO:teuthology.orchestra.run.smithi090.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-02T15:25:47.890 INFO:teuthology.orchestra.run.smithi120.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-06-02T15:25:47.895 INFO:teuthology.orchestra.run.smithi135.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 123 kB/s | 35 kB 00:00 2024-06-02T15:25:47.908 INFO:teuthology.orchestra.run.smithi090.stdout:(136/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-06-02T15:25:47.932 INFO:teuthology.orchestra.run.smithi120.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-02T15:25:47.934 INFO:teuthology.orchestra.run.smithi090.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-06-02T15:25:47.946 INFO:teuthology.orchestra.run.smithi135.stdout:(81/159): python3-mako-1.1.4-6.el9.noarch.rpm 338 kB/s | 172 kB 00:00 2024-06-02T15:25:47.959 INFO:teuthology.orchestra.run.smithi090.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 369 kB/s | 31 kB 00:00 2024-06-02T15:25:47.966 INFO:teuthology.orchestra.run.smithi120.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-06-02T15:25:48.009 INFO:teuthology.orchestra.run.smithi090.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 791 kB/s | 59 kB 00:00 2024-06-02T15:25:48.035 INFO:teuthology.orchestra.run.smithi090.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-06-02T15:25:48.068 INFO:teuthology.orchestra.run.smithi090.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-02T15:25:48.093 INFO:teuthology.orchestra.run.smithi090.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 615 kB/s | 51 kB 00:00 2024-06-02T15:25:48.127 INFO:teuthology.orchestra.run.smithi090.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-02T15:25:48.150 INFO:teuthology.orchestra.run.smithi120.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 92 kB/s | 20 kB 00:00 2024-06-02T15:25:48.175 INFO:teuthology.orchestra.run.smithi120.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 918 kB/s | 192 kB 00:00 2024-06-02T15:25:48.186 INFO:teuthology.orchestra.run.smithi090.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-06-02T15:25:48.212 INFO:teuthology.orchestra.run.smithi090.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 237 kB/s | 20 kB 00:00 2024-06-02T15:25:48.234 INFO:teuthology.orchestra.run.smithi120.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-06-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi090.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.7 MB/s | 427 kB 00:00 2024-06-02T15:25:48.330 INFO:teuthology.orchestra.run.smithi090.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-06-02T15:25:48.397 INFO:teuthology.orchestra.run.smithi090.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-06-02T15:25:48.446 INFO:teuthology.orchestra.run.smithi135.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 501 kB/s | 250 kB 00:00 2024-06-02T15:25:48.713 INFO:teuthology.orchestra.run.smithi135.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 1.7 MB/s | 6.0 MB 00:03 2024-06-02T15:25:48.722 INFO:teuthology.orchestra.run.smithi090.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 7.1 MB/s | 2.3 MB 00:00 2024-06-02T15:25:48.759 INFO:teuthology.orchestra.run.smithi120.stdout:(148/159): librbd1-19.0.0-3882.gb7328fed.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-06-02T15:25:48.881 INFO:teuthology.orchestra.run.smithi090.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-06-02T15:25:48.940 INFO:teuthology.orchestra.run.smithi090.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 719 kB/s | 41 kB 00:00 2024-06-02T15:25:48.947 INFO:teuthology.orchestra.run.smithi135.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 444 kB/s | 222 kB 00:00 2024-06-02T15:25:49.032 INFO:teuthology.orchestra.run.smithi090.stdout:(152/159): librados2-19.0.0-3882.gb7328fed.el9. 4.3 MB/s | 3.4 MB 00:00 2024-06-02T15:25:49.055 INFO:teuthology.orchestra.run.smithi135.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 226 kB/s | 77 kB 00:00 2024-06-02T15:25:49.068 INFO:teuthology.orchestra.run.smithi120.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-06-02T15:25:49.123 INFO:teuthology.orchestra.run.smithi090.stdout:(153/159): librbd1-19.0.0-3882.gb7328fed.el9.x8 4.0 MB/s | 3.1 MB 00:00 2024-06-02T15:25:49.157 INFO:teuthology.orchestra.run.smithi090.stdout:(154/159): NetworkManager-tui-1.47.91-1.el9.x86 1.1 MB/s | 248 kB 00:00 2024-06-02T15:25:49.182 INFO:teuthology.orchestra.run.smithi090.stdout:(155/159): elfutils-debuginfod-client-0.191-4.e 245 kB/s | 37 kB 00:00 2024-06-02T15:25:49.184 INFO:teuthology.orchestra.run.smithi120.stdout:(150/159): librados2-19.0.0-3882.gb7328fed.el9. 3.4 MB/s | 3.4 MB 00:01 2024-06-02T15:25:49.198 INFO:teuthology.orchestra.run.smithi135.stdout:(86/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-06-02T15:25:49.224 INFO:teuthology.orchestra.run.smithi090.stdout:(156/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.1 MB/s | 207 kB 00:00 2024-06-02T15:25:49.226 INFO:teuthology.orchestra.run.smithi120.stdout:(151/159): NetworkManager-libnm-1.47.91-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-06-02T15:25:49.302 INFO:teuthology.orchestra.run.smithi120.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 3.2 MB/s | 248 kB 00:00 2024-06-02T15:25:49.314 INFO:teuthology.orchestra.run.smithi135.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 115 kB/s | 42 kB 00:00 2024-06-02T15:25:49.352 INFO:teuthology.orchestra.run.smithi120.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 735 kB/s | 37 kB 00:00 2024-06-02T15:25:49.378 INFO:teuthology.orchestra.run.smithi120.stdout:(154/159): NetworkManager-team-1.47.91-1.el9.x8 214 kB/s | 41 kB 00:00 2024-06-02T15:25:49.400 INFO:teuthology.orchestra.run.smithi090.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-06-02T15:25:49.411 INFO:teuthology.orchestra.run.smithi120.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.4 MB/s | 207 kB 00:00 2024-06-02T15:25:49.495 INFO:teuthology.orchestra.run.smithi120.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-06-02T15:25:49.508 INFO:teuthology.orchestra.run.smithi090.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 1.3 MB 00:00 2024-06-02T15:25:49.533 INFO:teuthology.orchestra.run.smithi090.stdout:(159/159): elfutils-libs-0.191-4.el9.x86_64.rpm 693 kB/s | 260 kB 00:00 2024-06-02T15:25:49.545 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-06-02T15:25:49.545 INFO:teuthology.orchestra.run.smithi090.stdout:Total 11 MB/s | 227 MB 00:19 2024-06-02T15:25:49.615 INFO:teuthology.orchestra.run.smithi135.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 381 kB/s | 159 kB 00:00 2024-06-02T15:25:49.640 INFO:teuthology.orchestra.run.smithi135.stdout:(89/159): python3-protobuf-3.14.0-13.el9.noarch 461 kB/s | 269 kB 00:00 2024-06-02T15:25:49.891 INFO:teuthology.orchestra.run.smithi135.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 484 kB/s | 279 kB 00:00 2024-06-02T15:25:49.946 INFO:teuthology.orchestra.run.smithi120.stdout:(157/159): ceph-test-19.0.0-3882.gb7328fed.el9. 4.6 MB/s | 49 MB 00:10 2024-06-02T15:25:49.949 INFO:teuthology.orchestra.run.smithi135.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 153 kB/s | 51 kB 00:00 2024-06-02T15:25:49.971 INFO:teuthology.orchestra.run.smithi120.stdout:(158/159): elfutils-libs-0.191-4.el9.x86_64.rpm 438 kB/s | 260 kB 00:00 2024-06-02T15:25:49.974 INFO:teuthology.orchestra.run.smithi135.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 161 kB/s | 54 kB 00:00 2024-06-02T15:25:50.029 INFO:teuthology.orchestra.run.smithi120.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 2.4 MB 00:00 2024-06-02T15:25:50.033 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-06-02T15:25:50.033 INFO:teuthology.orchestra.run.smithi120.stdout:Total 11 MB/s | 227 MB 00:20 2024-06-02T15:25:50.208 INFO:teuthology.orchestra.run.smithi135.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-06-02T15:25:50.309 INFO:teuthology.orchestra.run.smithi135.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 132 kB/s | 44 kB 00:00 2024-06-02T15:25:50.584 INFO:teuthology.orchestra.run.smithi135.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 130 kB/s | 36 kB 00:00 2024-06-02T15:25:50.726 INFO:teuthology.orchestra.run.smithi135.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 427 kB/s | 221 kB 00:00 2024-06-02T15:25:50.918 INFO:teuthology.orchestra.run.smithi135.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 142 kB/s | 47 kB 00:00 2024-06-02T15:25:51.229 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-06-02T15:25:51.235 INFO:teuthology.orchestra.run.smithi135.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 599 kB/s | 305 kB 00:00 2024-06-02T15:25:51.269 INFO:teuthology.orchestra.run.smithi135.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 165 kB/s | 58 kB 00:00 2024-06-02T15:25:51.383 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-06-02T15:25:51.383 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-06-02T15:25:51.619 INFO:teuthology.orchestra.run.smithi135.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 244 kB/s | 94 kB 00:00 2024-06-02T15:25:51.644 INFO:teuthology.orchestra.run.smithi135.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-06-02T15:25:51.728 INFO:teuthology.orchestra.run.smithi135.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-06-02T15:25:51.862 INFO:teuthology.orchestra.run.smithi135.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-06-02T15:25:51.962 INFO:teuthology.orchestra.run.smithi135.stdout:(104/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 131 kB/s | 45 kB 00:00 2024-06-02T15:25:52.138 INFO:teuthology.orchestra.run.smithi135.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-06-02T15:25:52.180 INFO:teuthology.orchestra.run.smithi135.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-06-02T15:25:52.222 INFO:teuthology.orchestra.run.smithi135.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-06-02T15:25:52.280 INFO:teuthology.orchestra.run.smithi135.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-06-02T15:25:52.339 INFO:teuthology.orchestra.run.smithi135.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-06-02T15:25:52.398 INFO:teuthology.orchestra.run.smithi135.stdout:(110/159): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-06-02T15:25:52.423 INFO:teuthology.orchestra.run.smithi135.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 803 kB/s | 67 kB 00:00 2024-06-02T15:25:52.457 INFO:teuthology.orchestra.run.smithi135.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-02T15:25:52.524 INFO:teuthology.orchestra.run.smithi135.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-06-02T15:25:52.566 INFO:teuthology.orchestra.run.smithi135.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 839 kB 00:00 2024-06-02T15:25:52.591 INFO:teuthology.orchestra.run.smithi135.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 649 kB/s | 43 kB 00:00 2024-06-02T15:25:52.617 INFO:teuthology.orchestra.run.smithi135.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-06-02T15:25:52.642 INFO:teuthology.orchestra.run.smithi135.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-06-02T15:25:52.676 INFO:teuthology.orchestra.run.smithi135.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-06-02T15:25:52.701 INFO:teuthology.orchestra.run.smithi135.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-02T15:25:52.735 INFO:teuthology.orchestra.run.smithi135.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-06-02T15:25:52.769 INFO:teuthology.orchestra.run.smithi135.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-06-02T15:25:52.782 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-06-02T15:25:52.827 INFO:teuthology.orchestra.run.smithi135.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-06-02T15:25:52.870 INFO:teuthology.orchestra.run.smithi135.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-06-02T15:25:52.887 INFO:teuthology.orchestra.run.smithi135.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 181 kB/s | 11 kB 00:00 2024-06-02T15:25:52.912 INFO:teuthology.orchestra.run.smithi135.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 420 kB/s | 18 kB 00:00 2024-06-02T15:25:52.937 INFO:teuthology.orchestra.run.smithi135.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-06-02T15:25:52.963 INFO:teuthology.orchestra.run.smithi135.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-06-02T15:25:52.988 INFO:teuthology.orchestra.run.smithi135.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-06-02T15:25:53.030 INFO:teuthology.orchestra.run.smithi135.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-06-02T15:25:53.032 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-06-02T15:25:53.032 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-06-02T15:25:53.055 INFO:teuthology.orchestra.run.smithi135.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-06-02T15:25:53.097 INFO:teuthology.orchestra.run.smithi135.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-06-02T15:25:53.123 INFO:teuthology.orchestra.run.smithi135.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 857 kB/s | 58 kB 00:00 2024-06-02T15:25:53.157 INFO:teuthology.orchestra.run.smithi135.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-02T15:25:53.182 INFO:teuthology.orchestra.run.smithi135.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-06-02T15:25:53.203 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-06-02T15:25:53.205 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-06-02T15:25:53.207 INFO:teuthology.orchestra.run.smithi135.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-02T15:25:53.233 INFO:teuthology.orchestra.run.smithi135.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-06-02T15:25:53.258 INFO:teuthology.orchestra.run.smithi135.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-06-02T15:25:53.292 INFO:teuthology.orchestra.run.smithi135.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-06-02T15:25:53.317 INFO:teuthology.orchestra.run.smithi135.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-06-02T15:25:53.342 INFO:teuthology.orchestra.run.smithi135.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-06-02T15:25:53.368 INFO:teuthology.orchestra.run.smithi135.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-02T15:25:53.401 INFO:teuthology.orchestra.run.smithi135.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-06-02T15:25:53.552 INFO:teuthology.orchestra.run.smithi135.stdout:(143/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-06-02T15:25:53.577 INFO:teuthology.orchestra.run.smithi135.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 428 kB/s | 90 kB 00:00 2024-06-02T15:25:53.603 INFO:teuthology.orchestra.run.smithi135.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-06-02T15:25:53.636 INFO:teuthology.orchestra.run.smithi135.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-06-02T15:25:53.679 INFO:teuthology.orchestra.run.smithi135.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-06-02T15:25:53.706 INFO:teuthology.orchestra.run.smithi135.stdout:(148/159): python3-zc-lockfile-2.0-10.el9.noarc 132 kB/s | 20 kB 00:00 2024-06-02T15:25:53.988 INFO:teuthology.orchestra.run.smithi135.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-06-02T15:25:54.096 INFO:teuthology.orchestra.run.smithi135.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-06-02T15:25:54.147 INFO:teuthology.orchestra.run.smithi135.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 826 kB/s | 41 kB 00:00 2024-06-02T15:25:54.205 INFO:teuthology.orchestra.run.smithi135.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-06-02T15:25:54.256 INFO:teuthology.orchestra.run.smithi135.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 734 kB/s | 37 kB 00:00 2024-06-02T15:25:54.314 INFO:teuthology.orchestra.run.smithi135.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.4 MB/s | 207 kB 00:00 2024-06-02T15:25:54.373 INFO:teuthology.orchestra.run.smithi135.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-06-02T15:25:54.448 INFO:teuthology.orchestra.run.smithi135.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-06-02T15:25:54.557 INFO:teuthology.orchestra.run.smithi135.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-06-02T15:25:55.649 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-06-02T15:25:55.649 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-06-02T15:25:55.741 INFO:teuthology.orchestra.run.smithi135.stdout:(158/159): librados2-19.0.0-3882.gb7328fed.el9. 1.6 MB/s | 3.4 MB 00:02 2024-06-02T15:25:55.791 INFO:teuthology.orchestra.run.smithi135.stdout:(159/159): librbd1-19.0.0-3882.gb7328fed.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-06-02T15:25:55.794 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-06-02T15:25:55.794 INFO:teuthology.orchestra.run.smithi135.stdout:Total 7.6 MB/s | 227 MB 00:30 2024-06-02T15:25:57.046 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-06-02T15:25:57.203 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-06-02T15:25:57.203 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-06-02T15:25:59.078 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-06-02T15:25:59.078 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-06-02T15:25:59.399 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-06-02T15:25:59.484 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T15:25:59.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T15:25:59.575 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T15:25:59.610 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T15:25:59.637 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T15:25:59.678 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T15:26:00.000 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T15:26:00.105 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:00.210 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:00.240 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:00.330 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:26:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T15:26:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T15:26:00.641 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T15:26:00.669 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:00.773 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:00.796 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:00.858 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:00.880 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:00.902 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:01.073 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:01.195 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:01.377 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:01.545 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:01.630 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:01.855 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T15:26:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T15:26:02.043 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-06-02T15:26:02.108 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-06-02T15:26:02.143 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-06-02T15:26:02.187 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-06-02T15:26:02.219 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-06-02T15:26:02.256 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-06-02T15:26:02.323 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-06-02T15:26:02.377 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-06-02T15:26:02.408 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-06-02T15:26:02.444 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-06-02T15:26:02.483 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-06-02T15:26:02.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-06-02T15:26:02.554 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 33/170 2024-06-02T15:26:02.589 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:26:02.599 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T15:26:02.618 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 35/170 2024-06-02T15:26:02.679 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T15:26:02.690 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-06-02T15:26:02.733 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-06-02T15:26:02.804 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T15:26:02.808 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-06-02T15:26:02.869 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T15:26:02.904 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T15:26:02.939 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-06-02T15:26:02.995 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-06-02T15:26:03.039 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T15:26:03.456 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T15:26:03.505 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-06-02T15:26:03.577 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:03.581 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-06-02T15:26:03.618 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-06-02T15:26:03.655 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-06-02T15:26:03.704 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-06-02T15:26:03.742 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 46/170 2024-06-02T15:26:03.784 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-06-02T15:26:03.805 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:03.837 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:03.943 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-06-02T15:26:03.967 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:04.013 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:26:04.019 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-06-02T15:26:04.076 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T15:26:04.110 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-06-02T15:26:04.167 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-06-02T15:26:04.240 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T15:26:04.271 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T15:26:04.308 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T15:26:04.354 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T15:26:04.381 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T15:26:04.398 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T15:26:04.442 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T15:26:04.443 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T15:26:04.469 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:04.483 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T15:26:04.619 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T15:26:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:04.654 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:04.660 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T15:26:04.702 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T15:26:04.740 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:04.772 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:04.796 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T15:26:04.836 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T15:26:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:04.878 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T15:26:04.909 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T15:26:04.934 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T15:26:05.007 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T15:26:05.135 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T15:26:05.166 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T15:26:05.171 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:05.202 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T15:26:05.237 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T15:26:05.276 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T15:26:05.304 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-06-02T15:26:05.307 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T15:26:05.346 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T15:26:05.380 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T15:26:05.405 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T15:26:05.413 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T15:26:05.443 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T15:26:05.455 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T15:26:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:05.489 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T15:26:05.496 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T15:26:05.531 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T15:26:05.556 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T15:26:05.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T15:26:05.601 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T15:26:05.628 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T15:26:05.659 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T15:26:05.706 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T15:26:05.804 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T15:26:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:05.861 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T15:26:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:05.937 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T15:26:06.050 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:06.090 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:06.166 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:06.203 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:06.295 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:06.324 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:26:06.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T15:26:06.563 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T15:26:06.581 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T15:26:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:06.622 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T15:26:06.650 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:06.741 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T15:26:06.754 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:06.775 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T15:26:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T15:26:06.784 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T15:26:06.830 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 15/170 2024-06-02T15:26:06.852 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:06.874 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T15:26:06.877 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T15:26:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-06-02T15:26:06.921 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T15:26:06.966 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T15:26:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-06-02T15:26:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-06-02T15:26:07.053 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T15:26:07.077 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:07.086 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T15:26:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-06-02T15:26:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-06-02T15:26:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-06-02T15:26:07.217 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T15:26:07.261 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T15:26:07.300 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T15:26:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-06-02T15:26:07.358 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:07.377 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T15:26:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-06-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T15:26:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-06-02T15:26:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-06-02T15:26:07.518 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:07.610 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-06-02T15:26:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 33/170 2024-06-02T15:26:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:26:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 35/170 2024-06-02T15:26:07.807 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T15:26:07.876 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-06-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T15:26:07.940 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T15:26:07.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-06-02T15:26:07.975 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-06-02T15:26:08.016 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T15:26:08.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-06-02T15:26:08.047 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-06-02T15:26:08.091 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-06-02T15:26:08.131 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T15:26:08.134 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-06-02T15:26:08.167 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-06-02T15:26:08.195 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-06-02T15:26:08.225 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-06-02T15:26:08.263 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-06-02T15:26:08.296 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-06-02T15:26:08.317 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-06-02T15:26:08.356 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-06-02T15:26:08.408 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-06-02T15:26:08.493 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-06-02T15:26:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 33/170 2024-06-02T15:26:08.570 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:26:08.600 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 35/170 2024-06-02T15:26:08.672 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-06-02T15:26:08.714 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-06-02T15:26:08.789 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-06-02T15:26:08.932 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-06-02T15:26:08.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-06-02T15:26:09.008 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-06-02T15:26:09.110 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-06-02T15:26:09.146 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-06-02T15:26:09.195 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-06-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-06-02T15:26:09.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 46/170 2024-06-02T15:26:09.356 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T15:26:09.393 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-06-02T15:26:09.445 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T15:26:09.472 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T15:26:09.506 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T15:26:09.526 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-06-02T15:26:09.613 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-06-02T15:26:09.639 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-06-02T15:26:09.650 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-06-02T15:26:09.686 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-06-02T15:26:09.735 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-06-02T15:26:09.737 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-06-02T15:26:09.773 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 46/170 2024-06-02T15:26:09.814 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-06-02T15:26:09.818 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T15:26:09.868 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:09.874 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-06-02T15:26:09.959 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-06-02T15:26:09.960 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-06-02T15:26:09.960 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:09.990 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 104/170 2024-06-02T15:26:10.017 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-06-02T15:26:10.026 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T15:26:10.082 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T15:26:10.110 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-06-02T15:26:10.120 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T15:26:10.158 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T15:26:10.173 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-06-02T15:26:10.228 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T15:26:10.246 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T15:26:10.277 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T15:26:10.277 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T15:26:10.314 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T15:26:10.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T15:26:10.360 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T15:26:10.405 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T15:26:10.407 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T15:26:10.481 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T15:26:10.521 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T15:26:10.524 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T15:26:10.555 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:10.614 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T15:26:10.627 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:10.661 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T15:26:10.663 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T15:26:10.665 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 108/170 2024-06-02T15:26:10.700 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T15:26:10.733 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T15:26:10.741 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T15:26:10.838 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T15:26:10.864 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T15:26:10.875 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T15:26:10.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T15:26:10.923 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T15:26:10.940 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T15:26:10.965 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T15:26:10.986 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T15:26:11.021 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T15:26:11.040 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T15:26:11.054 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T15:26:11.167 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T15:26:11.197 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T15:26:11.225 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T15:26:11.235 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T15:26:11.259 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T15:26:11.299 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T15:26:11.339 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T15:26:11.377 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T15:26:11.411 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T15:26:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T15:26:11.445 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T15:26:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T15:26:11.486 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T15:26:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T15:26:11.521 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T15:26:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T15:26:11.608 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T15:26:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T15:26:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T15:26:11.691 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T15:26:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T15:26:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T15:26:11.746 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T15:26:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T15:26:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T15:26:11.892 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T15:26:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T15:26:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T15:26:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T15:26:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T15:26:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T15:26:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T15:26:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T15:26:12.672 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T15:26:12.714 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T15:26:12.814 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T15:26:12.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T15:26:12.914 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T15:26:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:13.003 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T15:26:13.050 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T15:26:13.235 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T15:26:13.273 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T15:26:13.350 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T15:26:13.488 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T15:26:13.501 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:13.547 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T15:26:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T15:26:13.639 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:13.639 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T15:26:13.639 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T15:26:13.639 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:13.665 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T15:26:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T15:26:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T15:26:13.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T15:26:13.996 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T15:26:14.023 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T15:26:14.130 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T15:26:14.238 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T15:26:14.276 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T15:26:14.413 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T15:26:15.232 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T15:26:15.376 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T15:26:15.393 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T15:26:15.477 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T15:26:15.504 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T15:26:15.545 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T15:26:15.859 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T15:26:15.909 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:16.000 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:16.038 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 104/170 2024-06-02T15:26:16.074 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T15:26:16.557 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T15:26:16.587 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:16.658 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:16.697 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 108/170 2024-06-02T15:26:16.980 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T15:26:17.133 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T15:26:17.166 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T15:26:17.207 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T15:26:17.607 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T15:26:17.652 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:17.809 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 103/170 2024-06-02T15:26:17.842 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 104/170 2024-06-02T15:26:17.900 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T15:26:18.455 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T15:26:18.490 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:18.596 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 107/170 2024-06-02T15:26:18.626 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 108/170 2024-06-02T15:26:19.145 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:19.185 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:19.734 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:19.771 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:19.863 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:19.864 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T15:26:19.864 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T15:26:19.864 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:19.897 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:21.337 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:21.391 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:22.046 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 109/170 2024-06-02T15:26:22.096 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:22.248 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 110/170 2024-06-02T15:26:22.248 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T15:26:22.248 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T15:26:22.248 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:22.288 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-06-02T15:26:25.321 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:25.492 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:25.507 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:25.507 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:25.508 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T15:26:25.508 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:26.979 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:27.005 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:27.230 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:27.252 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:27.252 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:27.252 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T15:26:27.252 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:27.252 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:27.253 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:27.689 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:27.711 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:27.711 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:27.712 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T15:26:27.712 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:27.712 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:27.712 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:27.810 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T15:26:27.858 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T15:26:27.915 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T15:26:28.107 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T15:26:28.156 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T15:26:28.234 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T15:26:28.322 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T15:26:28.461 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T15:26:28.499 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T15:26:28.531 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T15:26:28.559 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T15:26:29.010 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:29.041 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:29.073 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 128/170 2024-06-02T15:26:29.134 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 129/170 2024-06-02T15:26:29.164 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:30.193 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:30.233 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:31.187 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:31.336 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:31.502 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 133/170 2024-06-02T15:26:31.535 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:31.558 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:31.605 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:31.614 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-06-02T15:26:31.696 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-06-02T15:26:31.697 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:31.711 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 136/170 2024-06-02T15:26:31.759 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 137/170 2024-06-02T15:26:31.898 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:31.914 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:31.914 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:31.914 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T15:26:31.914 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:32.687 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T15:26:32.722 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:32.745 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:32.745 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:32.745 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T15:26:32.745 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:32.746 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:32.746 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:32.790 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:32.812 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:32.812 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:32.812 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T15:26:32.812 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:32.857 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:32.879 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:32.880 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:32.880 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T15:26:32.880 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:32.880 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:32.880 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:33.401 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:33.427 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:33.660 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:33.682 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:33.683 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:33.683 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T15:26:33.683 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:33.683 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:33.683 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:34.136 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:34.158 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:34.158 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:34.158 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T15:26:34.158 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:34.158 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:34.159 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:34.266 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T15:26:34.324 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T15:26:34.379 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T15:26:34.567 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T15:26:34.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T15:26:34.699 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T15:26:34.796 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T15:26:34.937 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T15:26:34.979 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T15:26:35.011 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T15:26:35.047 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T15:26:35.483 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:35.520 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:35.552 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 128/170 2024-06-02T15:26:35.614 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 129/170 2024-06-02T15:26:35.651 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:36.482 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 111/170 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-06-02T15:26:36.483 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:36.694 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:36.729 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:36.735 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:36.782 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 112/170 2024-06-02T15:26:36.782 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:36.783 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T15:26:36.783 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:37.693 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:37.721 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:37.778 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 143/170 2024-06-02T15:26:37.819 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 144/170 2024-06-02T15:26:37.834 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:37.860 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 145/170 2024-06-02T15:26:37.893 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T15:26:37.920 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T15:26:37.990 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 133/170 2024-06-02T15:26:38.031 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:38.053 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:38.054 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:38.054 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T15:26:38.054 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:38.054 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:38.054 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:38.090 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 148/170 2024-06-02T15:26:38.107 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:38.118 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:38.158 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T15:26:38.191 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 150/170 2024-06-02T15:26:38.215 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 136/170 2024-06-02T15:26:38.231 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T15:26:38.261 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T15:26:38.272 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 137/170 2024-06-02T15:26:38.316 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:26:38.449 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:38.526 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 113/170 2024-06-02T15:26:38.527 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:38.527 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T15:26:38.527 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:38.527 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T15:26:38.527 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:38.774 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 114/170 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T15:26:38.840 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:38.870 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:38.906 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:38.925 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:39.197 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T15:26:39.225 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:39.248 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:39.248 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:39.248 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T15:26:39.248 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:39.249 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:39.249 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:39.301 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:39.323 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:39.324 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:39.324 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T15:26:39.324 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:39.325 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:39.368 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:39.391 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:39.391 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:39.391 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T15:26:39.391 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:39.391 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:39.392 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:39.392 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 115/170 2024-06-02T15:26:39.392 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:39.392 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T15:26:39.393 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:39.393 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T15:26:39.393 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:39.526 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T15:26:39.588 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T15:26:39.689 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T15:26:39.703 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:39.726 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:39.754 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 155/170 2024-06-02T15:26:39.845 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T15:26:39.886 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T15:26:39.939 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T15:26:39.968 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T15:26:39.998 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T15:26:40.016 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T15:26:40.040 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:26:40.066 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T15:26:40.085 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T15:26:40.092 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T15:26:40.092 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:40.177 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:40.201 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T15:26:40.202 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:40.202 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T15:26:40.286 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:40.311 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T15:26:40.316 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:40.316 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:40.374 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T15:26:40.428 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T15:26:40.479 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T15:26:40.496 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:40.497 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:40.514 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T15:26:40.589 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:40.622 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T15:26:40.686 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:41.180 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:41.218 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 127/170 2024-06-02T15:26:41.271 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 128/170 2024-06-02T15:26:41.372 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 129/170 2024-06-02T15:26:41.405 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:42.715 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 130/170 2024-06-02T15:26:42.766 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:43.872 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 131/170 2024-06-02T15:26:43.908 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:44.066 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 132/170 2024-06-02T15:26:44.247 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 133/170 2024-06-02T15:26:44.285 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:44.331 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 143/170 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 134/170 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T15:26:44.354 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:44.372 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 144/170 2024-06-02T15:26:44.411 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:44.422 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 145/170 2024-06-02T15:26:44.447 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 135/170 2024-06-02T15:26:44.462 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T15:26:44.489 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T15:26:44.590 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 136/170 2024-06-02T15:26:44.665 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 137/170 2024-06-02T15:26:44.677 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 148/170 2024-06-02T15:26:44.727 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T15:26:44.759 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 150/170 2024-06-02T15:26:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T15:26:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T15:26:44.876 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:26:45.466 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:45.507 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:45.529 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:45.709 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T15:26:45.741 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:45.810 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 139/170 2024-06-02T15:26:45.811 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:45.811 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T15:26:45.811 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:45.811 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T15:26:45.811 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:45.867 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:45.937 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3882.gb7328 140/170 2024-06-02T15:26:45.937 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:45.938 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T15:26:45.938 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:46.043 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:46.118 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 141/170 2024-06-02T15:26:46.118 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:46.118 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T15:26:46.118 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:46.118 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T15:26:46.119 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:46.369 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:46.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 1/170 2024-06-02T15:26:46.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2/170 2024-06-02T15:26:46.369 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 3/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 4/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 5/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 6/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 7/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 11/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 12/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 13/170 2024-06-02T15:26:46.370 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 15/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 16/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 17/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 18/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 19/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 20/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 21/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 22/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 23/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 24/170 2024-06-02T15:26:46.371 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 25/170 2024-06-02T15:26:46.372 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 155/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 26/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 27/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 28/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 29/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 30/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 31/170 2024-06-02T15:26:46.385 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 32/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 33/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 35/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 36/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 37/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 38/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 39/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T15:26:46.386 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T15:26:46.387 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T15:26:46.388 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T15:26:46.389 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T15:26:46.390 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T15:26:46.391 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T15:26:46.392 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T15:26:46.393 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T15:26:46.394 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 149/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 151/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T15:26:46.395 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T15:26:46.396 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T15:26:46.461 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T15:26:46.493 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:46.502 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 142/170 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T15:26:46.566 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:26:46.584 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T15:26:46.631 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T15:26:46.656 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:26:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T15:26:46.708 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T15:26:46.708 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:46.792 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:46.825 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T15:26:46.825 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:46.910 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:46.935 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T15:26:46.940 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:46.940 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:47.120 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:47.120 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:47.213 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:47.246 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T15:26:47.310 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:48.894 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T15:26:48.895 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.896 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:48.897 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.898 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T15:26:48.899 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T15:26:48.900 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T15:26:48.901 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T15:26:48.902 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T15:26:48.903 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T15:26:48.904 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T15:26:48.905 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T15:26:48.906 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-06-02T15:26:48.907 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-06-02T15:26:49.343 DEBUG:teuthology.parallel:result is None 2024-06-02T15:26:51.667 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 143/170 2024-06-02T15:26:51.734 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 144/170 2024-06-02T15:26:51.805 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 145/170 2024-06-02T15:26:51.846 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T15:26:51.884 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T15:26:52.168 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 148/170 2024-06-02T15:26:52.226 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T15:26:52.260 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 150/170 2024-06-02T15:26:52.334 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T15:26:52.384 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T15:26:52.435 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:26:53.193 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 1/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 3/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 4/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 5/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 6/170 2024-06-02T15:26:53.194 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 7/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 11/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 12/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 13/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 15/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 16/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 17/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 18/170 2024-06-02T15:26:53.195 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 19/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 20/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 21/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 22/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 23/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 24/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 25/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 26/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 27/170 2024-06-02T15:26:53.196 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 28/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 29/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 30/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 31/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 32/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 33/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 35/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 36/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 37/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 38/170 2024-06-02T15:26:53.197 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 39/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T15:26:53.198 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T15:26:53.199 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T15:26:53.200 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T15:26:53.201 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T15:26:53.202 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T15:26:53.203 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T15:26:53.204 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T15:26:53.205 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 149/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 151/170 2024-06-02T15:26:53.206 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T15:26:53.207 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T15:26:53.208 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T15:26:53.372 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:53.425 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:53.482 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T15:26:54.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 155/170 2024-06-02T15:26:54.970 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T15:26:55.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T15:26:55.192 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T15:26:55.246 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T15:26:55.282 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:26:55.317 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T15:26:55.359 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T15:26:55.359 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:55.500 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T15:26:55.554 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T15:26:55.555 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:55.691 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T15:26:55.730 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T15:26:55.780 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:55.780 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:56.000 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T15:26:56.000 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:56.000 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-06-02T15:26:56.000 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:56.000 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-06-02T15:26:56.001 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.002 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.003 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T15:26:56.004 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.005 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T15:26:56.006 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T15:26:56.007 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T15:26:56.008 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T15:26:56.009 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T15:26:56.010 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.011 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T15:26:56.012 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T15:26:56.013 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T15:26:56.014 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T15:26:56.014 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:26:56.014 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-06-02T15:26:56.118 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T15:26:56.118 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:56.257 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T15:26:56.310 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T15:26:56.440 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:56.467 DEBUG:teuthology.parallel:result is None 2024-06-02T15:26:56.759 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T15:26:56.759 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-06-02T15:26:56.759 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:27:04.025 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 1/170 2024-06-02T15:27:04.025 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2/170 2024-06-02T15:27:04.025 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 3/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 4/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3882.gb7328 5/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 6/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 7/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 8/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 9/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 10/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 11/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 12/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 13/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 14/170 2024-06-02T15:27:04.026 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_ 15/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 16/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 17/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_6 18/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86 19/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 20/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el 21/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3882.gb7328fed.el9. 22/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_6 23/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 24/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 25/170 2024-06-02T15:27:04.027 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 26/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 27/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 28/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 29/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed. 30/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noa 31/170 2024-06-02T15:27:04.028 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.n 32/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3882.gb73 33/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el 34/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 35/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.e 36/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 37/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 38/170 2024-06-02T15:27:04.041 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 39/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T15:27:04.042 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T15:27:04.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T15:27:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T15:27:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T15:27:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T15:27:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T15:27:04.044 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T15:27:04.045 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T15:27:04.046 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T15:27:04.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T15:27:04.048 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T15:27:04.049 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T15:27:04.050 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T15:27:04.051 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T15:27:04.052 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T15:27:04.053 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T15:27:04.054 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 149/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 151/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T15:27:04.055 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T15:27:04.056 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T15:27:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T15:27:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:27:06.244 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-06-02T15:27:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T15:27:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T15:27:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T15:27:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T15:27:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T15:27:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T15:27:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T15:27:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T15:27:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T15:27:06.246 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-3882.gb7328fed.el9.noarch 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T15:27:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T15:27:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T15:27:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T15:27:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T15:27:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T15:27:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T15:27:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T15:27:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3882.gb7328fed.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T15:27:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:27:06.260 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-06-02T15:27:06.921 DEBUG:teuthology.parallel:result is None 2024-06-02T15:27:06.922 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:07.071 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T15:27:07.128 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.129 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.129 INFO:teuthology.task.install:The correct ceph version 19.0.0-3882.gb7328fed is installed. 2024-06-02T15:27:07.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:07.274 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T15:27:07.300 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.300 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.300 INFO:teuthology.task.install:The correct ceph version 19.0.0-3882.gb7328fed is installed. 2024-06-02T15:27:07.303 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:07.515 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T15:27:07.540 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.540 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3882.gb7328fed.el9 2024-06-02T15:27:07.541 INFO:teuthology.task.install:The correct ceph version 19.0.0-3882.gb7328fed is installed. 2024-06-02T15:27:07.542 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-02T15:27:07.543 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:27:07.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T15:27:07.610 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:27:07.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T15:27:07.641 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:27:07.641 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T15:27:07.673 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-02T15:27:07.673 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:27:07.673 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T15:27:07.735 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T15:27:07.838 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:27:07.838 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T15:27:07.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T15:27:07.935 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:27:07.935 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T15:27:07.964 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T15:27:08.031 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-02T15:27:08.031 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:27:08.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T15:27:08.095 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T15:27:08.197 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:27:08.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T15:27:08.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T15:27:08.284 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:27:08.285 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T15:27:08.309 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T15:27:08.372 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-02T15:27:08.373 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:27:08.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T15:27:08.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T15:27:08.541 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:27:08.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T15:27:08.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T15:27:08.628 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:27:08.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T15:27:08.652 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T15:27:08.715 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-02T15:27:08.815 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504', 'subvols': {'create': 2}} 2024-06-02T15:27:08.815 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:08.815 INFO:tasks.cephadm:Cluster fsid is 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:27:08.816 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-02T15:27:08.816 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.b': '172.21.15.120', 'mon.c': '172.21.15.135'} 2024-06-02T15:27:08.816 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-06-02T15:27:08.816 INFO:tasks.cephadm:First mgr is x 2024-06-02T15:27:08.816 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-02T15:27:08.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-06-02T15:27:08.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname $(hostname -s) 2024-06-02T15:27:08.910 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-06-02T15:27:08.933 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-02T15:27:08.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:09.080 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b7328fed0e2e9ab66b09f8134f3cebfc71214504', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55203/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3882-gb7328fed', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3882.gb7328fed'}, 'url': 'https://3.chacra.ceph.com/r/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-22 17:15:03.629717', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-02T15:27:09.306 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref main, sha1 b7328fed0e2e9ab66b09f8134f3cebfc71214504 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:27:09.308 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/x86_64/flavors/default/cephadm 2024-06-02T15:27:09.308 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/x86_64/flavors/default/cephadm 2024-06-02T15:27:09.308 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:09.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 782793 Jun 2 15:27 /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:09.975 DEBUG:teuthology.orchestra.run.smithi120:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:10.823 INFO:teuthology.orchestra.run.smithi120.stdout:-rw-r--r--. 1 ubuntu ubuntu 782793 Jun 2 15:27 /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:10.823 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/b7328fed0e2e9ab66b09f8134f3cebfc71214504/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:11.267 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-r--r--. 1 ubuntu ubuntu 782793 Jun 2 15:27 /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:11.267 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:11.328 DEBUG:teuthology.orchestra.run.smithi120:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:11.347 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T15:27:11.380 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 on all hosts... 2024-06-02T15:27:11.381 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 pull 2024-06-02T15:27:11.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 pull 2024-06-02T15:27:11.391 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 pull 2024-06-02T15:27:12.204 INFO:teuthology.orchestra.run.smithi120.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504... 2024-06-02T15:27:12.214 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504... 2024-06-02T15:27:12.662 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504... 2024-06-02T15:28:15.755 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-02T15:28:15.756 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_version": "ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev)", 2024-06-02T15:28:15.756 INFO:teuthology.orchestra.run.smithi120.stdout: "image_id": "57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d", 2024-06-02T15:28:15.756 INFO:teuthology.orchestra.run.smithi120.stdout: "repo_digests": [ 2024-06-02T15:28:15.756 INFO:teuthology.orchestra.run.smithi120.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a" 2024-06-02T15:28:15.756 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-06-02T15:28:15.757 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-02T15:28:20.596 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev)", 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d", 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a" 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-06-02T15:28:20.597 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-02T15:28:26.111 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:28:26.111 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev)", 2024-06-02T15:28:26.111 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d", 2024-06-02T15:28:26.112 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-06-02T15:28:26.112 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a" 2024-06-02T15:28:26.112 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:28:26.112 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:28:26.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-06-02T15:28:26.267 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph 2024-06-02T15:28:26.302 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-06-02T15:28:26.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-06-02T15:28:26.424 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /etc/ceph 2024-06-02T15:28:26.453 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-06-02T15:28:26.483 INFO:tasks.cephadm:Writing seed config... 2024-06-02T15:28:26.484 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-02T15:28:26.484 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-02T15:28:26.485 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-02T15:28:26.486 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-02T15:28:26.487 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-02T15:28:26.488 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-02T15:28:26.489 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-02T15:28:26.490 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-02T15:28:26.490 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-02T15:28:26.490 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-02T15:28:26.490 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-02T15:28:26.492 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:28:26.492 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-02T15:28:26.551 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 935be7dc-20f4-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-02T15:28:26.551 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service 2024-06-02T15:28:26.557 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-02T15:28:26.605 INFO:tasks.cephadm:Bootstrapping... 2024-06-02T15:28:26.606 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 -v bootstrap --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:28:26.875 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-06-02T15:28:26.876 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504', '-v', 'bootstrap', '--fsid', '935be7dc-20f4-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-06-02T15:28:26.876 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-02T15:28:26.876 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-06-02T15:28:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.0.2 2024-06-02T15:28:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-06-02T15:28:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-06-02T15:28:26.953 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-02T15:28:26.953 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-02T15:28:26.971 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-02T15:28:26.971 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:28:26.993 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-06-02T15:28:27.011 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-06-02T15:28:27.011 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-06-02T15:28:27.012 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-06-02T15:28:27.067 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.0.2 2024-06-02T15:28:27.068 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-02T15:28:27.068 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-06-02T15:28:27.068 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-06-02T15:28:27.084 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-02T15:28:27.084 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-02T15:28:27.099 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-02T15:28:27.099 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:28:27.116 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-06-02T15:28:27.135 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-06-02T15:28:27.135 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-06-02T15:28:27.136 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-06-02T15:28:27.136 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:27.136 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 140415156760096 on /run/cephadm/935be7dc-20f4-11ef-bc9b-c7b262605968.lock 2024-06-02T15:28:27.137 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 140415156760096 acquired on /run/cephadm/935be7dc-20f4-11ef-bc9b-c7b262605968.lock 2024-06-02T15:28:27.138 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-06-02T15:28:27.139 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-06-02T15:28:27.140 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-06-02T15:28:27.153 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-06-02T15:28:27.154 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-06-02T15:28:27.165 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-02T15:28:27.165 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-02T15:28:27.165 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-06-02T15:28:27.174 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-02T15:28:27.175 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-02T15:28:27.175 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-02T15:28:27.175 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-02T15:28:27.176 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-06-02T15:28:27.177 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-02T15:28:27.178 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-06-02T15:28:27.178 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-06-02T15:28:27.178 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-02T15:28:27.181 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-02T15:28:27.183 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504... 2024-06-02T15:28:27.841 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d 2024-06-02T15:28:27.841 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504... 2024-06-02T15:28:27.842 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-06-02T15:28:27.842 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:af2bfe2195989993dd601658f16ff4b7ee99cf335930517d8b15388a4a91c26b 2024-06-02T15:28:27.842 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-06-02T15:28:27.842 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d 2024-06-02T15:28:27.843 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-02T15:28:29.055 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev) 2024-06-02T15:28:29.055 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev) 2024-06-02T15:28:29.055 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-06-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-06-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-06-02T15:28:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCej1xmteIKIhAAtpnq4n4v5AB5w3ddjZNeAw== 2024-06-02T15:28:32.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCfj1xmfihwLBAAVAoaGAnA7dtn2t7v5iUNag== 2024-06-02T15:28:33.533 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQCgj1xmA6TTMhAAnyOSDD9LevXWWfuWCnFIcw== 2024-06-02T15:28:33.535 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-06-02T15:28:34.643 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:34.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-02T15:28:34.645 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:28:34.645 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-06-02T15:28:35.813 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-06-02T15:28:36.198 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-02T15:28:36.431 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-02T15:28:36.718 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target. 2024-06-02T15:28:36.718 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target. 2024-06-02T15:28:37.077 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a 2024-06-02T15:28:37.077 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-02T15:28:37.364 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target.wants/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@.service. 2024-06-02T15:28:38.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:37 smithi090 podman[31206]: 2024-06-02 15:28:37.662805443 +0000 UTC m=+0.031158701 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:28:38.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 podman[31206]: 2024-06-02 15:28:38.031932056 +0000 UTC m=+0.400285347 container init 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=main-b7328fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-06-02T15:28:38.151 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T15:28:38.152 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-06-02T15:28:38.168 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T15:28:38.168 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:28:38.168 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-06-02T15:28:38.168 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-06-02T15:28:38.169 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-06-02T15:28:38.169 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-06-02T15:28:38.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 podman[31206]: 2024-06-02 15:28:38.042423128 +0000 UTC m=+0.410776393 container start 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe) 2024-06-02T15:28:38.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 ceph-mon[31225]: mkfs 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:38.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 bash[31206]: 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a 2024-06-02T15:28:38.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 ceph-mon[31225]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T15:28:38.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:38 smithi090 systemd[1]: Started Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: monmap epoch 1 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: last_changed 2024-06-02T15:28:33.985461+0000 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:28:39.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: min_mon_release 19 (squid) 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: election_strategy: 1 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: fsmap 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: osdmap e1: 0 total, 0 up, 0 in 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: mgrmap e1: no daemons active 2024-06-02T15:28:39.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:39 smithi090 ceph-mon[31225]: from='client.? 172.21.15.90:0/1251745494' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-02T15:28:39.478 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.751881s) 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-02T15:28:39.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.777+0000 7f420be00700 1 Processor -- start 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.777+0000 7f420be00700 1 -- start start 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.778+0000 7f420be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.778+0000 7f420be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f420407d8d0 con 0x7f420407eea0 2024-06-02T15:28:39.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.779+0000 7f4209400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:39.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.779+0000 7f4209400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51792/0 (socket says 172.21.15.90:51792) 2024-06-02T15:28:39.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.779+0000 7f4209400700 1 -- 172.21.15.90:0/1055739914 learned_addr learned my addr 172.21.15.90:0/1055739914 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:39.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.780+0000 7f4209400700 1 -- 172.21.15.90:0/1055739914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420407da50 con 0x7f420407eea0 2024-06-02T15:28:39.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.780+0000 7f4209400700 1 --2- 172.21.15.90:0/1055739914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41f80098a0 tx=0x7f41f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdffe5317e7808 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:39.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.780+0000 7f4203e00700 1 -- 172.21.15.90:0/1055739914 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8004160 con 0x7f420407eea0 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.780+0000 7f4203e00700 1 -- 172.21.15.90:0/1055739914 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8004300 con 0x7f420407eea0 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.780+0000 7f4203e00700 1 -- 172.21.15.90:0/1055739914 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f80036a0 con 0x7f420407eea0 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 -- 172.21.15.90:0/1055739914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 msgr2=0x7f420407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 --2- 172.21.15.90:0/1055739914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f41f80098a0 tx=0x7f41f8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 -- 172.21.15.90:0/1055739914 shutdown_connections 2024-06-02T15:28:39.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 --2- 172.21.15.90:0/1055739914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420407d300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 -- 172.21.15.90:0/1055739914 >> 172.21.15.90:0/1055739914 conn(0x7f42041043b0 msgr2=0x7f42041067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 -- 172.21.15.90:0/1055739914 shutdown_connections 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.781+0000 7f420be00700 1 -- 172.21.15.90:0/1055739914 wait complete. 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.782+0000 7f420be00700 1 Processor -- start 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.782+0000 7f420be00700 1 -- start start 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f420be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f420be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f420407df80 con 0x7f420407eea0 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f4209400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f4209400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51794/0 (socket says 172.21.15.90:51794) 2024-06-02T15:28:39.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f4209400700 1 -- 172.21.15.90:0/1251745494 learned_addr learned my addr 172.21.15.90:0/1251745494 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.783+0000 7f4209400700 1 -- 172.21.15.90:0/1251745494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420419ee90 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4209400700 1 --2- 172.21.15.90:0/1251745494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000c00 tx=0x7f41f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8003db0 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8011630 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420419f120 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41f8011950 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041a1e10 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f41f8011af0 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.784+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f41f801c450 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.785+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4204064ba0 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.788+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f41f8032020 con 0x7f420407eea0 2024-06-02T15:28:39.490 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.867+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f420402e560 con 0x7f420407eea0 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.867+0000 7f4202000700 1 -- 172.21.15.90:0/1251745494 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f41f8004260 con 0x7f420407eea0 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.869+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 msgr2=0x7f420419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.869+0000 7f420be00700 1 --2- 172.21.15.90:0/1251745494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000c00 tx=0x7f41f8004750 comp rx=0 tx=0).stop 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.869+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 shutdown_connections 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.869+0000 7f420be00700 1 --2- 172.21.15.90:0/1251745494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f420407eea0 0x7f420419e950 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.869+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 >> 172.21.15.90:0/1251745494 conn(0x7f42041043b0 msgr2=0x7f4204105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.870+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 shutdown_connections 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:38.870+0000 7f420be00700 1 -- 172.21.15.90:0/1251745494 wait complete. 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-06-02T15:28:39.491 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-06-02T15:28:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:40 smithi090 ceph-mon[31225]: from='client.? 172.21.15.90:0/4007965970' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-02T15:28:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:40 smithi090 ceph-mon[31225]: from='client.? 172.21.15.90:0/4007965970' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-02T15:28:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-02T15:28:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.041+0000 7f8202a00700 1 Processor -- start 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.041+0000 7f8202a00700 1 -- start start 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.041+0000 7f8202a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.042+0000 7f8202a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc10b870 con 0x7f81fc10aef0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.042+0000 7f81fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.042+0000 7f81fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51806/0 (socket says 172.21.15.90:51806) 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.042+0000 7f81fbe00700 1 -- 172.21.15.90:0/2730787977 learned_addr learned my addr 172.21.15.90:0/2730787977 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.042+0000 7f81fbe00700 1 -- 172.21.15.90:0/2730787977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc10c0d0 con 0x7f81fc10aef0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.043+0000 7f81fbe00700 1 --2- 172.21.15.90:0/2730787977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098a0 tx=0x7f81ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb733dc227fe5db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.043+0000 7f81faa00700 1 -- 172.21.15.90:0/2730787977 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ec004160 con 0x7f81fc10aef0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.043+0000 7f81faa00700 1 -- 172.21.15.90:0/2730787977 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec004300 con 0x7f81fc10aef0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.043+0000 7f81faa00700 1 -- 172.21.15.90:0/2730787977 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ec0036a0 con 0x7f81fc10aef0 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 -- 172.21.15.90:0/2730787977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 msgr2=0x7f81fc10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 --2- 172.21.15.90:0/2730787977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0098a0 tx=0x7f81ec009bb0 comp rx=0 tx=0).stop 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 -- 172.21.15.90:0/2730787977 shutdown_connections 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 --2- 172.21.15.90:0/2730787977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc10b330 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 -- 172.21.15.90:0/2730787977 >> 172.21.15.90:0/2730787977 conn(0x7f81fc07e310 msgr2=0x7f81fc07e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 -- 172.21.15.90:0/2730787977 shutdown_connections 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.044+0000 7f8202a00700 1 -- 172.21.15.90:0/2730787977 wait complete. 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.045+0000 7f8202a00700 1 Processor -- start 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f8202a00700 1 -- start start 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f8202a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f8202a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc10c480 con 0x7f81fc10aef0 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f81fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f81fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51808/0 (socket says 172.21.15.90:51808) 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.046+0000 7f81fbe00700 1 -- 172.21.15.90:0/4007965970 learned_addr learned my addr 172.21.15.90:0/4007965970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f81fbe00700 1 -- 172.21.15.90:0/4007965970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc19eeb0 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f81fbe00700 1 --2- 172.21.15.90:0/4007965970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000c00 tx=0x7f81ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ec003db0 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec011630 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81ec011950 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc19f140 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.047+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc1a1e30 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.048+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f81ec01a460 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.048+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f81ec01b780 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.049+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc064ba0 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.051+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f81ec01b980 con 0x7f81fc10aef0 2024-06-02T15:28:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.128+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f81fc19f620 con 0x7f81fc10aef0 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.137+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f81fc19f620 con 0x7f81fc10aef0 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.138+0000 7f81f8c00700 1 -- 172.21.15.90:0/4007965970 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f81ec01a8e0 con 0x7f81fc10aef0 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 msgr2=0x7f81fc19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 --2- 172.21.15.90:0/4007965970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000c00 tx=0x7f81ec004750 comp rx=0 tx=0).stop 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 shutdown_connections 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 --2- 172.21.15.90:0/4007965970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc10aef0 0x7f81fc19e970 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 >> 172.21.15.90:0/4007965970 conn(0x7f81fc07e310 msgr2=0x7f81fc195270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 shutdown_connections 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:40.140+0000 7f8202a00700 1 -- 172.21.15.90:0/4007965970 wait complete. 2024-06-02T15:28:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-06-02T15:28:41.800 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:41 smithi090 ceph-mon[31225]: from='client.? 172.21.15.90:0/238804477' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:28:42.146 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.402+0000 7f04ec000700 1 Processor -- start 2024-06-02T15:28:42.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.402+0000 7f04ec000700 1 -- start start 2024-06-02T15:28:42.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.403+0000 7f04ec000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:42.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.403+0000 7f04ec000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4109760 con 0x7f04e4108d70 2024-06-02T15:28:42.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.403+0000 7f04e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:42.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.403+0000 7f04e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35426/0 (socket says 172.21.15.90:35426) 2024-06-02T15:28:42.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.403+0000 7f04e9600700 1 -- 172.21.15.90:0/3487979596 learned_addr learned my addr 172.21.15.90:0/3487979596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:42.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.404+0000 7f04e9600700 1 -- 172.21.15.90:0/3487979596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e4109f70 con 0x7f04e4108d70 2024-06-02T15:28:42.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.404+0000 7f04e9600700 1 --2- 172.21.15.90:0/3487979596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04d80098a0 tx=0x7f04d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32098b2ad8f9f20e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:42.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.404+0000 7f04e3e00700 1 -- 172.21.15.90:0/3487979596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8004160 con 0x7f04e4108d70 2024-06-02T15:28:42.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.404+0000 7f04e3e00700 1 -- 172.21.15.90:0/3487979596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04d8004300 con 0x7f04e4108d70 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.404+0000 7f04e3e00700 1 -- 172.21.15.90:0/3487979596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8011af0 con 0x7f04e4108d70 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 -- 172.21.15.90:0/3487979596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 msgr2=0x7f04e4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 --2- 172.21.15.90:0/3487979596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f04d80098a0 tx=0x7f04d8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 -- 172.21.15.90:0/3487979596 shutdown_connections 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 --2- 172.21.15.90:0/3487979596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e4109190 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 -- 172.21.15.90:0/3487979596 >> 172.21.15.90:0/3487979596 conn(0x7f04e41043b0 msgr2=0x7f04e41067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 -- 172.21.15.90:0/3487979596 shutdown_connections 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.405+0000 7f04ec000700 1 -- 172.21.15.90:0/3487979596 wait complete. 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04ec000700 1 Processor -- start 2024-06-02T15:28:42.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04ec000700 1 -- start start 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04ec000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04ec000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410d320 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04e9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35436/0 (socket says 172.21.15.90:35436) 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.407+0000 7f04e9600700 1 -- 172.21.15.90:0/238804477 learned_addr learned my addr 172.21.15.90:0/238804477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04e9600700 1 -- 172.21.15.90:0/238804477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e419d510 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04e9600700 1 --2- 172.21.15.90:0/238804477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8003b20 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04d801a8c0 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e419d7a0 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.408+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04d8024940 con 0x7f04e4108d70 2024-06-02T15:28:42.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.409+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a0490 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.409+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04d8022020 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.409+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04d802dd90 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.410+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e4064ba0 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.413+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f04d8042ec0 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.489+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f04e402e560 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.490+0000 7f04e2000700 1 -- 172.21.15.90:0/238804477 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f04d8031030 con 0x7f04e4108d70 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 msgr2=0x7f04e419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 --2- 172.21.15.90:0/238804477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f04d8000c00 tx=0x7f04d8011b70 comp rx=0 tx=0).stop 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 shutdown_connections 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 --2- 172.21.15.90:0/238804477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04e4108d70 0x7f04e419cfd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 >> 172.21.15.90:0/238804477 conn(0x7f04e41043b0 msgr2=0x7f04e4105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:42.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 shutdown_connections 2024-06-02T15:28:42.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:41.492+0000 7f04ec000700 1 -- 172.21.15.90:0/238804477 wait complete. 2024-06-02T15:28:42.152 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-06-02T15:28:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:42 smithi090 systemd[1]: Stopping Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:28:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a[31221]: 2024-06-02T15:28:48.991+0000 7fa465000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:28:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a[31221]: 2024-06-02T15:28:48.991+0000 7fa465000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-02T15:28:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:49 smithi090 podman[31469]: 2024-06-02 15:28:49.094366825 +0000 UTC m=+6.365529508 container died 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-06-02T15:28:54.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:54 smithi090 podman[31469]: 2024-06-02 15:28:54.510329546 +0000 UTC m=+11.781492258 container cleanup 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-06-02T15:28:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:54 smithi090 bash[31469]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a 2024-06-02T15:28:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:54 smithi090 podman[31482]: 2024-06-02 15:28:54.778005803 +0000 UTC m=+5.665624918 container remove 52fcb13881dc0f903331ac205641323b132967204ed983fbe0a10c68d7a1d12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-06-02T15:28:55.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-02T15:28:55.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 systemd[1]: Stopped Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:28:55.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.898s CPU time. 2024-06-02T15:28:55.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 systemd[1]: Starting Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:28:55.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 podman[31576]: 2024-06-02 15:28:55.62079677 +0000 UTC m=+0.125574799 container create f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:28:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 podman[31576]: 2024-06-02 15:28:55.529620503 +0000 UTC m=+0.034398495 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:28:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 podman[31576]: 2024-06-02 15:28:55.843643843 +0000 UTC m=+0.348421862 container init f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-02T15:28:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 podman[31576]: 2024-06-02 15:28:55.853827123 +0000 UTC m=+0.358605159 container start f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-06-02T15:28:55.957 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: set uid:gid to 167:167 (ceph:ceph) 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev), process ceph-mon, pid 2 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: pidfile_write: ignore empty --pid-file 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: load: jerasure load: lrc 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: RocksDB version: 7.9.2 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Git sha 0 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Compile date 2024-05-22 16:22:08 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: DB SUMMARY 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: DB Session ID: LQPVMUC3F6JQ3IP3I3IA 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: CURRENT file: CURRENT 2024-06-02T15:28:56.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: IDENTITY file: IDENTITY 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103516 ; 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.error_if_exists: 0 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.create_if_missing: 0 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.paranoid_checks: 1 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.env: 0x5590a310ccc0 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.fs: PosixFileSystem 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.info_log: 0x5590b64bdde0 2024-06-02T15:28:56.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_file_opening_threads: 16 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.statistics: (nil) 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.use_fsync: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_log_file_size: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.keep_log_file_num: 1000 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.recycle_log_file_num: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_fallocate: 1 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_mmap_reads: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_mmap_writes: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.use_direct_reads: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.create_missing_column_families: 0 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.db_log_dir: 2024-06-02T15:28:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.wal_dir: 2024-06-02T15:28:56.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-02T15:28:56.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-02T15:28:56.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-02T15:28:56.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.advise_random_on_open: 1 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.db_write_buffer_size: 0 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.write_buffer_manager: 0x5590b648f0e0 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.rate_limiter: (nil) 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.wal_recovery_mode: 2 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enable_thread_tracking: 0 2024-06-02T15:28:56.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enable_pipelined_write: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.unordered_write: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.row_cache: None 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.wal_filter: None 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_ingest_behind: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.two_write_queues: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.manual_wal_flush: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.wal_compression: 0 2024-06-02T15:28:56.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.atomic_flush: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.log_readahead_size: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.best_efforts_recovery: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.allow_data_in_errors: 0 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.db_host_id: __hostname__ 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_background_jobs: 2 2024-06-02T15:28:56.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_background_compactions: -1 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_subcompactions: 1 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_total_wal_size: 0 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_open_files: -1 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bytes_per_sync: 0 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-02T15:28:56.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_readahead_size: 0 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_background_flushes: -1 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Compression algorithms supported: 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kZSTD supported: 0 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kXpressCompression supported: 0 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kLZ4HCCompression supported: 1 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kZlibCompression supported: 1 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kSnappyCompression supported: 1 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kLZ4Compression supported: 1 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: kBZip2Compression supported: 0 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-02T15:28:56.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.merge_operator: 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_filter: None 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_filter_factory: None 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.sst_partitioner_factory: None 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5590b64bdfa0) 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-02T15:28:56.167 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x5590b64c11f0 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-06-02T15:28:56.168 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-06-02T15:28:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.write_buffer_size: 33554432 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_write_buffer_number: 2 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression: NoCompression 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression: Disabled 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.prefix_extractor: nullptr 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.num_levels: 7 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-02T15:28:56.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-02T15:28:56.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-02T15:28:56.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-02T15:28:56.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-02T15:28:56.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-02T15:28:56.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.level: 32767 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.strategy: 0 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-02T15:28:56.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.enabled: false 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.target_file_size_base: 67108864 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-02T15:28:56.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.arena_block_size: 1048576 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-02T15:28:56.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.disable_auto_compactions: 0 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.inplace_update_support: 0 2024-06-02T15:28:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.bloom_locality: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.max_successive_merges: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.paranoid_file_checks: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.force_consistency_checks: 1 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.report_bg_io_stats: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.ttl: 2592000 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-02T15:28:56.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enable_blob_files: false 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.min_blob_size: 0 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_file_size: 268435456 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.blob_file_starting_level: 0 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7e6ebdce-0894-4821-a2a0-62ed74159072 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342135908351, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-02T15:28:56.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342135910437, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98769, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96722, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13786, "raw_average_key_size": 49, "raw_value_size": 89205, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717342135, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7e6ebdce-0894-4821-a2a0-62ed74159072", "db_session_id": "LQPVMUC3F6JQ3IP3I3IA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342135910591, "job": 1, "event": "recovery_finished"} 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5590b65de000 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: DB pointer 0x5590b65ca000 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T15:28:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 98.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 98.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 67.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T15:28:56.179 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 67.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 13.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 13.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-02T15:28:56.180 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x5590b64c11f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???) e1 preinit fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).mds e1 new map 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).mds e1 print_map 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-06-02T15:28:56.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 bash[31576]: f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 systemd[1]: Started Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: monmap epoch 1 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: last_changed 2024-06-02T15:28:33.985461+0000 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: min_mon_release 19 (squid) 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: election_strategy: 1 2024-06-02T15:28:56.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:28:56.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: fsmap 2024-06-02T15:28:56.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: osdmap e1: 0 total, 0 up, 0 in 2024-06-02T15:28:56.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:55 smithi090 ceph-mon[31595]: mgrmap e1: no daemons active 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.686+0000 7fb5e4400700 1 Processor -- start 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.686+0000 7fb5e4400700 1 -- start start 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.686+0000 7fb5e4400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.686+0000 7fb5e4400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc109760 con 0x7fb5dc108d70 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.687+0000 7fb5e1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:57.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.687+0000 7fb5e1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40532/0 (socket says 172.21.15.90:40532) 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.687+0000 7fb5e1a00700 1 -- 172.21.15.90:0/2383520060 learned_addr learned my addr 172.21.15.90:0/2383520060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.688+0000 7fb5e1a00700 1 -- 172.21.15.90:0/2383520060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc109f70 con 0x7fb5dc108d70 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.688+0000 7fb5e1a00700 1 --2- 172.21.15.90:0/2383520060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00098a0 tx=0x7fb5d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376db273285627 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.688+0000 7fb5dbe00700 1 -- 172.21.15.90:0/2383520060 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0004160 con 0x7fb5dc108d70 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.688+0000 7fb5dbe00700 1 -- 172.21.15.90:0/2383520060 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb5d0004300 con 0x7fb5dc108d70 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.688+0000 7fb5dbe00700 1 -- 172.21.15.90:0/2383520060 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d0011af0 con 0x7fb5dc108d70 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.689+0000 7fb5e4400700 1 -- 172.21.15.90:0/2383520060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 msgr2=0x7fb5dc109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 --2- 172.21.15.90:0/2383520060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00098a0 tx=0x7fb5d0009bb0 comp rx=0 tx=0).stop 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 -- 172.21.15.90:0/2383520060 shutdown_connections 2024-06-02T15:28:57.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 --2- 172.21.15.90:0/2383520060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc109190 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 -- 172.21.15.90:0/2383520060 >> 172.21.15.90:0/2383520060 conn(0x7fb5dc1043b0 msgr2=0x7fb5dc1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 -- 172.21.15.90:0/2383520060 shutdown_connections 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.690+0000 7fb5e4400700 1 -- 172.21.15.90:0/2383520060 wait complete. 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.691+0000 7fb5e4400700 1 Processor -- start 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.691+0000 7fb5e4400700 1 -- start start 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.691+0000 7fb5e4400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.692+0000 7fb5e4400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc10d320 con 0x7fb5dc108d70 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.692+0000 7fb5e1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.692+0000 7fb5e1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40544/0 (socket says 172.21.15.90:40544) 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.692+0000 7fb5e1a00700 1 -- 172.21.15.90:0/3399394434 learned_addr learned my addr 172.21.15.90:0/3399394434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:28:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.692+0000 7fb5e1a00700 1 -- 172.21.15.90:0/3399394434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc19f730 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5e1a00700 1 --2- 172.21.15.90:0/3399394434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003f40 tx=0x7fb5d0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d00038b0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc19f9c0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb5d0003a50 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5d001a5c0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.693+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc1a26b0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.694+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb5d001ac50 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.694+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb5d0019700 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.694+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc064ba0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.698+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb5d0019900 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.774+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb5dc02e560 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.777+0000 7fb5da000700 1 -- 172.21.15.90:0/3399394434 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb5d0019ae0 con 0x7fb5dc108d70 2024-06-02T15:28:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 msgr2=0x7fb5dc19f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 --2- 172.21.15.90:0/3399394434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003f40 tx=0x7fb5d0011890 comp rx=0 tx=0).stop 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 shutdown_connections 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 --2- 172.21.15.90:0/3399394434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5dc108d70 0x7fb5dc19f1f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 >> 172.21.15.90:0/3399394434 conn(0x7fb5dc1043b0 msgr2=0x7fb5dc105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.778+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 shutdown_connections 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:28:56.779+0000 7fb5e4400700 1 -- 172.21.15.90:0/3399394434 wait complete. 2024-06-02T15:28:57.421 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-02T15:28:57.447 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:28:57.447 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-06-02T15:28:57.447 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-02T15:28:57.448 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-02T15:28:57.824 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x 2024-06-02T15:28:57.825 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-02T15:28:58.040 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:28:57 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3399394434' entity='client.admin' 2024-06-02T15:28:58.089 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968.target.wants/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@.service. 2024-06-02T15:28:58.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:58 smithi090 systemd[1]: Starting Ceph mgr.x for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:28:58.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:58 smithi090 podman[31821]: 2024-06-02 15:28:58.387871601 +0000 UTC m=+0.030036847 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 2024-06-02T15:28:58.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:58 smithi090 podman[31821]: 2024-06-02 15:28:58.495178887 +0000 UTC m=+0.137344170 container create 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-06-02T15:28:59.552 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T15:28:59.552 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-06-02T15:28:59.566 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T15:28:59.567 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:28:59.567 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-06-02T15:28:59.567 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-06-02T15:28:59.586 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T15:28:59.586 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-06-02T15:28:59.601 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T15:28:59.601 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:28:59.601 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-06-02T15:28:59.602 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-02T15:28:59.602 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-06-02T15:28:59.602 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-06-02T15:28:59.680 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 podman[31821]: 2024-06-02 15:28:59.431319359 +0000 UTC m=+1.073484649 container init 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-02T15:28:59.680 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 podman[31821]: 2024-06-02 15:28:59.441902067 +0000 UTC m=+1.084067342 container start 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-06-02T15:28:59.681 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 bash[31821]: 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 2024-06-02T15:28:59.681 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 systemd[1]: Started Ceph mgr.x for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:28:59.940 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:28:59.681+0000 7f46e7c8f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T15:29:00.231 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:28:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:28:59.945+0000 7f46e7c8f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:00.231 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:00.078+0000 7f46e7c8f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T15:29:00.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:00 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3181446638' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T15:29:00.773 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:00.493+0000 7f46e7c8f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T15:29:00.773 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:00.614+0000 7f46e7c8f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T15:29:01.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:01.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:01.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:29:01.064 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T15:29:01.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T15:29:01.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T15:29:01.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T15:29:01.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.065 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T15:29:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T15:29:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-06-02T15:29:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T15:29:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T15:29:01.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:01.067 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-06-02T15:29:01.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T15:29:01.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:01.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T15:29:01.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T15:29:01.068 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T15:29:01.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:01.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T15:29:01.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T15:29:01.069 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T15:29:01.077 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T15:29:01.078 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T15:29:01.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T15:28:38.118562+0000", 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:01.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.231+0000 7f037f600700 1 Processor -- start 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.232+0000 7f037f600700 1 -- start start 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.232+0000 7f037f600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.232+0000 7f037f600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f037807d890 con 0x7f037807ee60 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.233+0000 7f037cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.233+0000 7f037cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40576/0 (socket says 172.21.15.90:40576) 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.233+0000 7f037cc00700 1 -- 172.21.15.90:0/4195399346 learned_addr learned my addr 172.21.15.90:0/4195399346 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.233+0000 7f037cc00700 1 -- 172.21.15.90:0/4195399346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037807da10 con 0x7f037807ee60 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.234+0000 7f037cc00700 1 --2- 172.21.15.90:0/4195399346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f036c0098a0 tx=0x7f036c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da074c74c23f68f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.234+0000 7f0377400700 1 -- 172.21.15.90:0/4195399346 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036c004160 con 0x7f037807ee60 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.234+0000 7f0377400700 1 -- 172.21.15.90:0/4195399346 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f036c004300 con 0x7f037807ee60 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.234+0000 7f0377400700 1 -- 172.21.15.90:0/4195399346 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036c011af0 con 0x7f037807ee60 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 -- 172.21.15.90:0/4195399346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 msgr2=0x7f037807d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:01.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 --2- 172.21.15.90:0/4195399346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f036c0098a0 tx=0x7f036c009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 -- 172.21.15.90:0/4195399346 shutdown_connections 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 --2- 172.21.15.90:0/4195399346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037807d2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 -- 172.21.15.90:0/4195399346 >> 172.21.15.90:0/4195399346 conn(0x7f03781043b0 msgr2=0x7f03781067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 -- 172.21.15.90:0/4195399346 shutdown_connections 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.235+0000 7f037f600700 1 -- 172.21.15.90:0/4195399346 wait complete. 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.236+0000 7f037f600700 1 Processor -- start 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.236+0000 7f037f600700 1 -- start start 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037f600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037f600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f037810f340 con 0x7f037807ee60 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40582/0 (socket says 172.21.15.90:40582) 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037cc00700 1 -- 172.21.15.90:0/3181446638 learned_addr learned my addr 172.21.15.90:0/3181446638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:01.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.237+0000 7f037cc00700 1 -- 172.21.15.90:0/3181446638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037819f680 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f037cc00700 1 --2- 172.21.15.90:0/3181446638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f036c011ac0 tx=0x7f036c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036c003b20 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f036c01a8c0 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037819f910 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036c0248d0 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.238+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03781a2600 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.239+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f036c022020 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.239+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f036c02dde0 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.240+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0378064ba0 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.242+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f036c042ec0 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.325+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f037802e560 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.325+0000 7f0375600700 1 -- 172.21.15.90:0/3181446638 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f036c031020 con 0x7f037807ee60 2024-06-02T15:29:01.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 msgr2=0x7f037819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 --2- 172.21.15.90:0/3181446638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f036c011ac0 tx=0x7f036c011ba0 comp rx=0 tx=0).stop 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 shutdown_connections 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 --2- 172.21.15.90:0/3181446638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f037807ee60 0x7f037819f140 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 >> 172.21.15.90:0/3181446638 conn(0x7f03781043b0 msgr2=0x7f0378104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 shutdown_connections 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:00.327+0000 7f037f600700 1 -- 172.21.15.90:0/3181446638 wait complete. 2024-06-02T15:29:01.084 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-06-02T15:29:02.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:02.185+0000 7f46e7c8f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T15:29:02.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:02.410+0000 7f46e7c8f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:03.008 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:02.629+0000 7f46e7c8f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T15:29:03.008 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:03.009+0000 7f46e7c8f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T15:29:03.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:03.135+0000 7f46e7c8f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T15:29:03.650 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:03.395+0000 7f46e7c8f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T15:29:03.650 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:03.651+0000 7f46e7c8f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:03 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/881377545' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T15:29:04.564 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:04.492+0000 7f46e7c8f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T15:29:04.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T15:29:04.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T15:29:04.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T15:29:04.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T15:29:04.589 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T15:29:04.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:04.591 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T15:28:38.118562+0000", 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.726+0000 7f388ce00700 1 Processor -- start 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.726+0000 7f388ce00700 1 -- start start 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.726+0000 7f388ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.726+0000 7f388ce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f388810b890 con 0x7f388810af10 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.727+0000 7f3885e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.727+0000 7f3885e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35060/0 (socket says 172.21.15.90:35060) 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.727+0000 7f3885e00700 1 -- 172.21.15.90:0/2925137272 learned_addr learned my addr 172.21.15.90:0/2925137272 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.727+0000 7f3885e00700 1 -- 172.21.15.90:0/2925137272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388810c0f0 con 0x7f388810af10 2024-06-02T15:29:04.592 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.727+0000 7f3885e00700 1 --2- 172.21.15.90:0/2925137272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68590656b7fbcbc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.728+0000 7f3884a00700 1 -- 172.21.15.90:0/2925137272 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878004160 con 0x7f388810af10 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.728+0000 7f3884a00700 1 -- 172.21.15.90:0/2925137272 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3878004300 con 0x7f388810af10 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.728+0000 7f3884a00700 1 -- 172.21.15.90:0/2925137272 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878011af0 con 0x7f388810af10 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 -- 172.21.15.90:0/2925137272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 msgr2=0x7f388810b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 --2- 172.21.15.90:0/2925137272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38780098a0 tx=0x7f3878009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 -- 172.21.15.90:0/2925137272 shutdown_connections 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 --2- 172.21.15.90:0/2925137272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388810af10 0x7f388810b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 -- 172.21.15.90:0/2925137272 >> 172.21.15.90:0/2925137272 conn(0x7f388807e300 msgr2=0x7f388807e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 -- 172.21.15.90:0/2925137272 shutdown_connections 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.729+0000 7f388ce00700 1 -- 172.21.15.90:0/2925137272 wait complete. 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.730+0000 7f388ce00700 1 Processor -- start 2024-06-02T15:29:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.730+0000 7f388ce00700 1 -- start start 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.730+0000 7f388ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.730+0000 7f388ce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f388810cf80 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f3885e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f3885e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35062/0 (socket says 172.21.15.90:35062) 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f3885e00700 1 -- 172.21.15.90:0/881377545 learned_addr learned my addr 172.21.15.90:0/881377545 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f3885e00700 1 -- 172.21.15.90:0/881377545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388819fd20 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f3885e00700 1 --2- 172.21.15.90:0/881377545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f388810ac40 tx=0x7f3878003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.731+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38780119c0 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f387801f070 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388819ffb0 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f387801b440 con 0x7f388819f3c0 2024-06-02T15:29:04.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a2a90 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f387801ba90 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.732+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f387802d730 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.733+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3868005360 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.735+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f387801ad30 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.813+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3868005920 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.814+0000 7f387ea00700 1 -- 172.21.15.90:0/881377545 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3878024470 con 0x7f388819f3c0 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.815+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 msgr2=0x7f388819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.815+0000 7f388ce00700 1 --2- 172.21.15.90:0/881377545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f388810ac40 tx=0x7f3878003960 comp rx=0 tx=0).stop 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.815+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 shutdown_connections 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.815+0000 7f388ce00700 1 --2- 172.21.15.90:0/881377545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f388819f3c0 0x7f388819f7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.815+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 >> 172.21.15.90:0/881377545 conn(0x7f388807e300 msgr2=0x7f38881087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.816+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 shutdown_connections 2024-06-02T15:29:04.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:03.816+0000 7f388ce00700 1 -- 172.21.15.90:0/881377545 wait complete. 2024-06-02T15:29:04.596 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-06-02T15:29:04.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:04.778+0000 7f46e7c8f200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-06-02T15:29:05.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:05.325+0000 7f46e7c8f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T15:29:05.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:05.450+0000 7f46e7c8f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T15:29:06.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:05.908+0000 7f46e7c8f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T15:29:06.623 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:06.358+0000 7f46e7c8f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T15:29:06.623 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:06.481+0000 7f46e7c8f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T15:29:06.988 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:06.806+0000 7f46e7c8f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T15:29:07.239 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:06.990+0000 7f46e7c8f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T15:29:07.239 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:07 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:07.121+0000 7f46e7c8f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T15:29:07.533 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:07 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/536394942' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T15:29:07.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:07 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:07.535+0000 7f46e7c8f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:07.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:07 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:07.659+0000 7f46e7c8f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T15:29:07.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T15:29:07.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T15:29:07.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T15:29:07.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T15:29:07.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T15:29:07.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T15:28:38.118562+0000", 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.232+0000 7fd3afe00700 1 Processor -- start 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.233+0000 7fd3afe00700 1 -- start start 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.233+0000 7fd3afe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:07.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.233+0000 7fd3afe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a807cf10 con 0x7fd3a8109880 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.234+0000 7fd3ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.234+0000 7fd3ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35064/0 (socket says 172.21.15.90:35064) 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.234+0000 7fd3ad400700 1 -- 172.21.15.90:0/3883294939 learned_addr learned my addr 172.21.15.90:0/3883294939 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.234+0000 7fd3ad400700 1 -- 172.21.15.90:0/3883294939 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a810c1b0 con 0x7fd3a8109880 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.235+0000 7fd3ad400700 1 --2- 172.21.15.90:0/3883294939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8023ffa8f63d6f30 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.235+0000 7fd3a7e00700 1 -- 172.21.15.90:0/3883294939 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c004160 con 0x7fd3a8109880 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.235+0000 7fd3a7e00700 1 -- 172.21.15.90:0/3883294939 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd39c004300 con 0x7fd3a8109880 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.235+0000 7fd3a7e00700 1 -- 172.21.15.90:0/3883294939 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c011af0 con 0x7fd3a8109880 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 -- 172.21.15.90:0/3883294939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 msgr2=0x7fd3a810bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 --2- 172.21.15.90:0/3883294939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0098a0 tx=0x7fd39c009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 -- 172.21.15.90:0/3883294939 shutdown_connections 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 --2- 172.21.15.90:0/3883294939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a810bc70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:07.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 -- 172.21.15.90:0/3883294939 >> 172.21.15.90:0/3883294939 conn(0x7fd3a8103270 msgr2=0x7fd3a8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 -- 172.21.15.90:0/3883294939 shutdown_connections 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.236+0000 7fd3afe00700 1 -- 172.21.15.90:0/3883294939 wait complete. 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.237+0000 7fd3afe00700 1 Processor -- start 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.237+0000 7fd3afe00700 1 -- start start 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3afe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3afe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a810d230 con 0x7fd3a8109880 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35080/0 (socket says 172.21.15.90:35080) 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3ad400700 1 -- 172.21.15.90:0/536394942 learned_addr learned my addr 172.21.15.90:0/536394942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.238+0000 7fd3ad400700 1 -- 172.21.15.90:0/536394942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a819f820 con 0x7fd3a8109880 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3ad400700 1 --2- 172.21.15.90:0/536394942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd39c011ac0 tx=0x7fd39c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c003b20 con 0x7fd3a8109880 2024-06-02T15:29:08.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd39c01a8c0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a819fab0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd39c0248d0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.239+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81a27a0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.240+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd39c022020 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.240+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd39c02d940 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.240+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8064ba0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.243+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd39c02d3f0 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.324+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd3a802e560 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.325+0000 7fd3a6000700 1 -- 172.21.15.90:0/536394942 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd39c01aa70 con 0x7fd3a8109880 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 msgr2=0x7fd3a819f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 --2- 172.21.15.90:0/536394942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd39c011ac0 tx=0x7fd39c011ba0 comp rx=0 tx=0).stop 2024-06-02T15:29:08.001 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 shutdown_connections 2024-06-02T15:29:08.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 --2- 172.21.15.90:0/536394942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3a8109880 0x7fd3a819f2e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:08.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 >> 172.21.15.90:0/536394942 conn(0x7fd3a8103270 msgr2=0x7fd3a8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:08.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.327+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 shutdown_connections 2024-06-02T15:29:08.002 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:07.328+0000 7fd3afe00700 1 -- 172.21.15.90:0/536394942 wait complete. 2024-06-02T15:29:08.002 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-06-02T15:29:08.718 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:08.434+0000 7f46e7c8f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T15:29:08.719 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:08.719+0000 7f46e7c8f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T15:29:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: Activating manager daemon x 2024-06-02T15:29:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: mgrmap e2: x(active, starting, since 0.0119589s) 2024-06-02T15:29:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T15:29:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T15:29:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: Manager daemon x is now available 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T15:29:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:08 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' 2024-06-02T15:29:10.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:09 smithi090 ceph-mon[31595]: from='mgr.14100 172.21.15.90:0/1030531245' entity='mgr.x' 2024-06-02T15:29:10.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:09 smithi090 ceph-mon[31595]: mgrmap e3: x(active, since 1.02081s) 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T15:29:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:11.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T15:29:11.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T15:29:11.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T15:29:11.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T15:29:11.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T15:29:11.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T15:29:11.644 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:11.645 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:11.646 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T15:28:38.118562+0000", 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.578+0000 7fcadd000700 1 Processor -- start 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.578+0000 7fcadd000700 1 -- start start 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.578+0000 7fcadd000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.578+0000 7fcadd000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80833c0 con 0x7fcad810fcb0 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.579+0000 7fcad7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.579+0000 7fcad7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35152/0 (socket says 172.21.15.90:35152) 2024-06-02T15:29:11.647 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.579+0000 7fcad7400700 1 -- 172.21.15.90:0/1027805636 learned_addr learned my addr 172.21.15.90:0/1027805636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.579+0000 7fcad7400700 1 -- 172.21.15.90:0/1027805636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81125e0 con 0x7fcad810fcb0 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.579+0000 7fcad7400700 1 --2- 172.21.15.90:0/1027805636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcac80098a0 tx=0x7fcac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40519b6c5a52bcf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcad6000700 1 -- 172.21.15.90:0/1027805636 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac8004160 con 0x7fcad810fcb0 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcad6000700 1 -- 172.21.15.90:0/1027805636 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcac8004300 con 0x7fcad810fcb0 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcad6000700 1 -- 172.21.15.90:0/1027805636 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac8011af0 con 0x7fcad810fcb0 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcadd000700 1 -- 172.21.15.90:0/1027805636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 msgr2=0x7fcad81120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcadd000700 1 --2- 172.21.15.90:0/1027805636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcac80098a0 tx=0x7fcac8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcadd000700 1 -- 172.21.15.90:0/1027805636 shutdown_connections 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcadd000700 1 --2- 172.21.15.90:0/1027805636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81120a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.580+0000 7fcadd000700 1 -- 172.21.15.90:0/1027805636 >> 172.21.15.90:0/1027805636 conn(0x7fcad807e140 msgr2=0x7fcad8080560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.581+0000 7fcadd000700 1 -- 172.21.15.90:0/1027805636 shutdown_connections 2024-06-02T15:29:11.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.581+0000 7fcadd000700 1 -- 172.21.15.90:0/1027805636 wait complete. 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcadd000700 1 Processor -- start 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcadd000700 1 -- start start 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcadd000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcadd000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81134e0 con 0x7fcad810fcb0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcad7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcad7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35168/0 (socket says 172.21.15.90:35168) 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.582+0000 7fcad7400700 1 -- 172.21.15.90:0/3770144516 learned_addr learned my addr 172.21.15.90:0/3770144516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcad7400700 1 -- 172.21.15.90:0/3770144516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81a3600 con 0x7fcad810fcb0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcad7400700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcac8011ac0 tx=0x7fcac8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac8003970 con 0x7fcad810fcb0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcac8003b10 con 0x7fcad810fcb0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81a3890 con 0x7fcad810fcb0 2024-06-02T15:29:11.649 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.583+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcac8024610 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.584+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81a6580 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.585+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8064ba0 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.585+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fcac801a430 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.585+0000 7fcacfe00700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 0x7fcabc0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.585+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcac80535e0 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.585+0000 7fcad6a00700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 0x7fcabc0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.586+0000 7fcad6a00700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 0x7fcabc0407c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcac4006fd0 tx=0x7fcac4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.592+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcac800b3e0 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.789+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fcac801a9e0 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.911+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcad81a6750 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.912+0000 7fcacfe00700 1 -- 172.21.15.90:0/3770144516 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcad81a6750 con 0x7fcad810fcb0 2024-06-02T15:29:11.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.916+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 msgr2=0x7fcabc0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.916+0000 7fcadd000700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 0x7fcabc0407c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcac4006fd0 tx=0x7fcac4006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 msgr2=0x7fcad81a30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcac8011ac0 tx=0x7fcac8011ba0 comp rx=0 tx=0).stop 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 shutdown_connections 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcabc03e300 0x7fcabc0407c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 --2- 172.21.15.90:0/3770144516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad810fcb0 0x7fcad81a30c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 >> 172.21.15.90:0/3770144516 conn(0x7fcad807e140 msgr2=0x7fcad807ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 shutdown_connections 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:10.917+0000 7fcadd000700 1 -- 172.21.15.90:0/3770144516 wait complete. 2024-06-02T15:29:11.651 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-06-02T15:29:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:11 smithi090 ceph-mon[31595]: mgrmap e4: x(active, since 2s) 2024-06-02T15:29:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3770144516' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T15:29:13.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:12 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2299594860' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-02T15:29:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-06-02T15:29:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.255+0000 7f4305000700 1 Processor -- start 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.255+0000 7f4305000700 1 -- start start 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.256+0000 7f4305000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.256+0000 7f4305000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f430010b740 con 0x7f430010ad50 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.256+0000 7f42ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.256+0000 7f42ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60504/0 (socket says 172.21.15.90:60504) 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.256+0000 7f42ff400700 1 -- 172.21.15.90:0/923248507 learned_addr learned my addr 172.21.15.90:0/923248507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.257+0000 7f42ff400700 1 -- 172.21.15.90:0/923248507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430010bf50 con 0x7f430010ad50 2024-06-02T15:29:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.257+0000 7f42ff400700 1 --2- 172.21.15.90:0/923248507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f42f00098a0 tx=0x7f42f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100f9bdfa146bf3b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.257+0000 7f42fe000700 1 -- 172.21.15.90:0/923248507 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0004160 con 0x7f430010ad50 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.257+0000 7f42fe000700 1 -- 172.21.15.90:0/923248507 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42f0004300 con 0x7f430010ad50 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.257+0000 7f42fe000700 1 -- 172.21.15.90:0/923248507 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0011af0 con 0x7f430010ad50 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 -- 172.21.15.90:0/923248507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 msgr2=0x7f430010b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 --2- 172.21.15.90:0/923248507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f42f00098a0 tx=0x7f42f0009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 -- 172.21.15.90:0/923248507 shutdown_connections 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 --2- 172.21.15.90:0/923248507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430010b170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 -- 172.21.15.90:0/923248507 >> 172.21.15.90:0/923248507 conn(0x7f430007e140 msgr2=0x7f430007e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 -- 172.21.15.90:0/923248507 shutdown_connections 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.258+0000 7f4305000700 1 -- 172.21.15.90:0/923248507 wait complete. 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.259+0000 7f4305000700 1 Processor -- start 2024-06-02T15:29:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f4305000700 1 -- start start 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f4305000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f4305000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f430010f300 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f42ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f42ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60520/0 (socket says 172.21.15.90:60520) 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.260+0000 7f42ff400700 1 -- 172.21.15.90:0/2299594860 learned_addr learned my addr 172.21.15.90:0/2299594860 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f42ff400700 1 -- 172.21.15.90:0/2299594860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430019f240 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f42ff400700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f42f00117b0 tx=0x7f42f0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0003af0 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f42f0020070 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42f0019630 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430019f4d0 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.261+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001a21b0 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.262+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f42f001a9c0 con 0x7f430010ad50 2024-06-02T15:29:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.262+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4300064ba0 con 0x7f430010ad50 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.263+0000 7f42f7e00700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 0x7f42e4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.263+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42f0053a60 con 0x7f430010ad50 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.264+0000 7f42fea00700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 0x7f42e4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.264+0000 7f42fea00700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 0x7f42e4040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006fd0 tx=0x7f42ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.269+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42f00172b0 con 0x7f430010ad50 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.499+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f43001a2380 con 0x7f430010ad50 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.502+0000 7f42f7e00700 1 -- 172.21.15.90:0/2299594860 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f43001a2380 con 0x7f430010ad50 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 msgr2=0x7f42e4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 0x7f42e4040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006fd0 tx=0x7f42ec006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 msgr2=0x7f430019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f42f00117b0 tx=0x7f42f0011b70 comp rx=0 tx=0).stop 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 shutdown_connections 2024-06-02T15:29:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f42e403e470 0x7f42e4040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:13.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 --2- 172.21.15.90:0/2299594860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010ad50 0x7f430019ed00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:13.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 >> 172.21.15.90:0/2299594860 conn(0x7f430007e140 msgr2=0x7f43001084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:13.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.505+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 shutdown_connections 2024-06-02T15:29:13.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:12.506+0000 7f4305000700 1 -- 172.21.15.90:0/2299594860 wait complete. 2024-06-02T15:29:13.123 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-06-02T15:29:14.330 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: ignoring --setuser ceph since I am not root 2024-06-02T15:29:14.331 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: ignoring --setgroup ceph since I am not root 2024-06-02T15:29:14.331 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:14.328+0000 7fa8b059e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T15:29:14.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/582941848' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-02T15:29:14.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:14.568+0000 7fa8b059e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.692+0000 7f0032c00700 1 Processor -- start 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.692+0000 7f0032c00700 1 -- start start 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.693+0000 7f0032c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.693+0000 7f0032c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f002c109530 con 0x7f002c108b40 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.693+0000 7f0031800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.693+0000 7f0031800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60530/0 (socket says 172.21.15.90:60530) 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.693+0000 7f0031800700 1 -- 172.21.15.90:0/789016384 learned_addr learned my addr 172.21.15.90:0/789016384 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.694+0000 7f0031800700 1 -- 172.21.15.90:0/789016384 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c109d40 con 0x7f002c108b40 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.694+0000 7f0031800700 1 --2- 172.21.15.90:0/789016384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f00200098a0 tx=0x7f0020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae40998d70037f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.694+0000 7f002be00700 1 -- 172.21.15.90:0/789016384 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0020004160 con 0x7f002c108b40 2024-06-02T15:29:14.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.694+0000 7f002be00700 1 -- 172.21.15.90:0/789016384 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0020004300 con 0x7f002c108b40 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.694+0000 7f002be00700 1 -- 172.21.15.90:0/789016384 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0020011af0 con 0x7f002c108b40 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 -- 172.21.15.90:0/789016384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 msgr2=0x7f002c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 --2- 172.21.15.90:0/789016384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f00200098a0 tx=0x7f0020009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 -- 172.21.15.90:0/789016384 shutdown_connections 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 --2- 172.21.15.90:0/789016384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c108f60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 -- 172.21.15.90:0/789016384 >> 172.21.15.90:0/789016384 conn(0x7f002c104160 msgr2=0x7f002c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 -- 172.21.15.90:0/789016384 shutdown_connections 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.695+0000 7f0032c00700 1 -- 172.21.15.90:0/789016384 wait complete. 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.696+0000 7f0032c00700 1 Processor -- start 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0032c00700 1 -- start start 2024-06-02T15:29:14.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0032c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0032c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f002c10d0f0 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0031800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0031800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60542/0 (socket says 172.21.15.90:60542) 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0031800700 1 -- 172.21.15.90:0/582941848 learned_addr learned my addr 172.21.15.90:0/582941848 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.697+0000 7f0031800700 1 -- 172.21.15.90:0/582941848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002c19f200 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.698+0000 7f0031800700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.698+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0020003b20 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.698+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f002001a8c0 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.698+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002c19f490 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.698+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0020024940 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.699+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002c1a2180 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.699+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f0020022020 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.700+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002c064ba0 con 0x7f002c108b40 2024-06-02T15:29:14.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.700+0000 7f002a000700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 0x7f00140408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.700+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0020053400 con 0x7f002c108b40 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.700+0000 7f0030e00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 0x7f00140408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.701+0000 7f0030e00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 0x7f00140408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.706+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f002001aa70 con 0x7f002c108b40 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:13.987+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f002c1a2350 con 0x7f002c108b40 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.040+0000 7f002a000700 1 -- 172.21.15.90:0/582941848 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f002c1a2350 con 0x7f002c108b40 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.044+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 msgr2=0x7f00140408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.044+0000 7f0032c00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 0x7f00140408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.044+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 msgr2=0x7f002c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0020000c00 tx=0x7f0020011b70 comp rx=0 tx=0).stop 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 shutdown_connections 2024-06-02T15:29:14.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f001403e420 0x7f00140408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:14.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 --2- 172.21.15.90:0/582941848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f002c108b40 0x7f002c19ecc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:14.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 >> 172.21.15.90:0/582941848 conn(0x7f002c104160 msgr2=0x7f002c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:14.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.045+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 shutdown_connections 2024-06-02T15:29:14.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:14.046+0000 7f0032c00700 1 -- 172.21.15.90:0/582941848 wait complete. 2024-06-02T15:29:14.958 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:14.699+0000 7fa8b059e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T15:29:15.244 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:15 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/582941848' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-02T15:29:15.244 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:15 smithi090 ceph-mon[31595]: mgrmap e5: x(active, since 5s) 2024-06-02T15:29:15.245 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:15 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:15.135+0000 7fa8b059e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T15:29:15.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:15 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:15.251+0000 7fa8b059e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T15:29:16.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1044921508' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-02T15:29:16.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:16.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-02T15:29:16.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T15:29:16.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-02T15:29:16.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.400+0000 7fcafb000700 1 Processor -- start 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.400+0000 7fcafb000700 1 -- start start 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.401+0000 7fcafb000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.401+0000 7fcafb000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf407ce90 con 0x7fcaf41073c0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.401+0000 7fcaf9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.401+0000 7fcaf9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60562/0 (socket says 172.21.15.90:60562) 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.401+0000 7fcaf9c00700 1 -- 172.21.15.90:0/2905856881 learned_addr learned my addr 172.21.15.90:0/2905856881 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.402+0000 7fcaf9c00700 1 -- 172.21.15.90:0/2905856881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf4107d20 con 0x7fcaf41073c0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.402+0000 7fcaf9c00700 1 --2- 172.21.15.90:0/2905856881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2be3e59a4ff7a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.402+0000 7fcaf3e00700 1 -- 172.21.15.90:0/2905856881 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8004160 con 0x7fcaf41073c0 2024-06-02T15:29:16.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcaf3e00700 1 -- 172.21.15.90:0/2905856881 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcae8004300 con 0x7fcaf41073c0 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcaf3e00700 1 -- 172.21.15.90:0/2905856881 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8011af0 con 0x7fcaf41073c0 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 -- 172.21.15.90:0/2905856881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 msgr2=0x7fcaf41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 --2- 172.21.15.90:0/2905856881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcae80098a0 tx=0x7fcae8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 -- 172.21.15.90:0/2905856881 shutdown_connections 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 --2- 172.21.15.90:0/2905856881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf41077e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 -- 172.21.15.90:0/2905856881 >> 172.21.15.90:0/2905856881 conn(0x7fcaf4102fe0 msgr2=0x7fcaf4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.403+0000 7fcafb000700 1 -- 172.21.15.90:0/2905856881 shutdown_connections 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.404+0000 7fcafb000700 1 -- 172.21.15.90:0/2905856881 wait complete. 2024-06-02T15:29:16.361 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.404+0000 7fcafb000700 1 Processor -- start 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcafb000700 1 -- start start 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcafb000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcafb000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4108d50 con 0x7fcaf41073c0 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcaf9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcaf9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60568/0 (socket says 172.21.15.90:60568) 2024-06-02T15:29:16.362 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.405+0000 7fcaf9c00700 1 -- 172.21.15.90:0/1044921508 learned_addr learned my addr 172.21.15.90:0/1044921508 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:16.363 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcaf9c00700 1 -- 172.21.15.90:0/1044921508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf419f580 con 0x7fcaf41073c0 2024-06-02T15:29:16.363 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcaf9c00700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcae8011ac0 tx=0x7fcae8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8003970 con 0x7fcaf41073c0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcae8003b10 con 0x7fcaf41073c0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf419f810 con 0x7fcaf41073c0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcae8024610 con 0x7fcaf41073c0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.406+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41a24b0 con 0x7fcaf41073c0 2024-06-02T15:29:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf4064ba0 con 0x7fcaf41073c0 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fcae801a430 con 0x7fcaf41073c0 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcaf2000700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 0x7fcadc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcae8053810 con 0x7fcaf41073c0 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcaf9200700 1 -- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 msgr2=0x7fcadc040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.408+0000 7fcaf9200700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 0x7fcadc040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:29:16.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.414+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcae800b3e0 con 0x7fcaf41073c0 2024-06-02T15:29:16.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.609+0000 7fcaf9200700 1 -- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 msgr2=0x7fcadc040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:16.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.609+0000 7fcaf9200700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 0x7fcadc040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T15:29:16.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.702+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcaf402e560 con 0x7fcaf41073c0 2024-06-02T15:29:16.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.703+0000 7fcaf2000700 1 -- 172.21.15.90:0/1044921508 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcae802cd80 con 0x7fcaf41073c0 2024-06-02T15:29:16.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 msgr2=0x7fcadc040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 0x7fcadc040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 msgr2=0x7fcaf419f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcae8011ac0 tx=0x7fcae8011ba0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 shutdown_connections 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7fcadc03e4c0 0x7fcadc040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 --2- 172.21.15.90:0/1044921508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaf41073c0 0x7fcaf419f040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 >> 172.21.15.90:0/1044921508 conn(0x7fcaf4102fe0 msgr2=0x7fcaf41958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 shutdown_connections 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:15.706+0000 7fcafb000700 1 -- 172.21.15.90:0/1044921508 wait complete. 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-06-02T15:29:16.369 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-06-02T15:29:16.927 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:16 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:16.858+0000 7fa8b059e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T15:29:17.302 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:17 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:17.088+0000 7fa8b059e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:17.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:17 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:17.302+0000 7fa8b059e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T15:29:18.055 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:17 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:17.682+0000 7fa8b059e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T15:29:18.056 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:17 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:17.803+0000 7fa8b059e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T15:29:18.307 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:18 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:18.056+0000 7fa8b059e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T15:29:18.307 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:18 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:18.308+0000 7fa8b059e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:19.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:19 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:19.103+0000 7fa8b059e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T15:29:19.873 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:19 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:19.387+0000 7fa8b059e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-06-02T15:29:20.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:19 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:19.940+0000 7fa8b059e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T15:29:20.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:20 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:20.069+0000 7fa8b059e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T15:29:20.873 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:20 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:20.530+0000 7fa8b059e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T15:29:21.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:20 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:20.977+0000 7fa8b059e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T15:29:21.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:21.102+0000 7fa8b059e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T15:29:21.735 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:21.426+0000 7fa8b059e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T15:29:21.735 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:21.606+0000 7fa8b059e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T15:29:22.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:21.736+0000 7fa8b059e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T15:29:22.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:22 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:22.134+0000 7fa8b059e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:22.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:22 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:22.258+0000 7fa8b059e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T15:29:23.303 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:23.016+0000 7fa8b059e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:23.303+0000 7fa8b059e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: Active manager daemon x restarted 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: Activating manager daemon x 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: osdmap e2: 0 total, 0 up, 0 in 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: mgrmap e6: x(active, starting, since 0.00763887s) 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T15:29:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T15:29:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T15:29:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:23 smithi090 ceph-mon[31595]: Manager daemon x is now available 2024-06-02T15:29:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: Found migration_current of "None". Setting to last migration. 2024-06-02T15:29:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T15:29:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:24 smithi090 ceph-mon[31595]: mgrmap e7: x(active, since 1.01348s) 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.928+0000 7f7bd5000700 1 Processor -- start 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.928+0000 7f7bd5000700 1 -- start start 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.928+0000 7f7bd5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.928+0000 7f7bd5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd007ce90 con 0x7f7bd01097f0 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.929+0000 7f7bcde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.929+0000 7f7bcde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60578/0 (socket says 172.21.15.90:60578) 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.929+0000 7f7bcde00700 1 -- 172.21.15.90:0/3864532529 learned_addr learned my addr 172.21.15.90:0/3864532529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:24.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.929+0000 7f7bcde00700 1 -- 172.21.15.90:0/3864532529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd010c120 con 0x7f7bd01097f0 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bcde00700 1 --2- 172.21.15.90:0/3864532529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00098a0 tx=0x7f7bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76ce38c1dee29fd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bcca00700 1 -- 172.21.15.90:0/3864532529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc0004160 con 0x7f7bd01097f0 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bcca00700 1 -- 172.21.15.90:0/3864532529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7bc0004300 con 0x7f7bd01097f0 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bcca00700 1 -- 172.21.15.90:0/3864532529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc0011af0 con 0x7f7bd01097f0 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bd5000700 1 -- 172.21.15.90:0/3864532529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 msgr2=0x7f7bd010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.930+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3864532529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00098a0 tx=0x7f7bc0009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.931+0000 7f7bd5000700 1 -- 172.21.15.90:0/3864532529 shutdown_connections 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.931+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3864532529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd010bbe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.931+0000 7f7bd5000700 1 -- 172.21.15.90:0/3864532529 >> 172.21.15.90:0/3864532529 conn(0x7f7bd0103280 msgr2=0x7f7bd01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.931+0000 7f7bd5000700 1 -- 172.21.15.90:0/3864532529 shutdown_connections 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.931+0000 7f7bd5000700 1 -- 172.21.15.90:0/3864532529 wait complete. 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.932+0000 7f7bd5000700 1 Processor -- start 2024-06-02T15:29:24.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.932+0000 7f7bd5000700 1 -- start start 2024-06-02T15:29:24.992 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.933+0000 7f7bd5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:24.992 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.933+0000 7f7bd5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010d0f0 con 0x7f7bd01097f0 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.933+0000 7f7bcde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.933+0000 7f7bcde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60588/0 (socket says 172.21.15.90:60588) 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.933+0000 7f7bcde00700 1 -- 172.21.15.90:0/3095416481 learned_addr learned my addr 172.21.15.90:0/3095416481 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bcde00700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd01095b0 con 0x7f7bd01097f0 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bcde00700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0003f40 tx=0x7f7bc0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc00038b0 con 0x7f7bd01097f0 2024-06-02T15:29:24.994 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7bc0003a50 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd01077a0 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bc001a5c0 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.934+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd0107ce0 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.935+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f7bc001a8c0 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.936+0000 7f7bc6a00700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.936+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bb4040ff0 con 0x7f7bb403e420 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.936+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bc0053a50 con 0x7f7bd01097f0 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.936+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:16.936+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:17.137+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:17.137+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:17.538+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:17.538+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:18.339+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:18.339+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:19.941+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:19.941+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:23.144+0000 7f7bcd400700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4260623570 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:23.144+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:23.312+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f7bc001ac00 con 0x7f7bd01097f0 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:23.313+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 msgr2=0x7f7bb40408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:23.313+0000 7f7bc6a00700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.318+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f7bc0055670 con 0x7f7bd01097f0 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.318+0000 7f7bc6a00700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 0x7f7bb4044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.318+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7bc001cec0 con 0x7f7bb4041d10 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.320+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 0x7f7bb4044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.323+0000 7f7bcd400700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 0x7f7bb4044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc003a10 tx=0x7f7bbc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.324+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7bc001cec0 con 0x7f7bb4041d10 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.333+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7bd0064ba0 con 0x7f7bb4041d10 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.333+0000 7f7bc6a00700 1 -- 172.21.15.90:0/3095416481 <== mgr.14120 v2:172.21.15.90:6800/1560565398 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7bd0064ba0 con 0x7f7bb4041d10 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 msgr2=0x7f7bb4044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 0x7f7bb4044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc003a10 tx=0x7f7bbc009290 comp rx=0 tx=0).stop 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 msgr2=0x7f7bd0109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0003f40 tx=0x7f7bc0011890 comp rx=0 tx=0).stop 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 shutdown_connections 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7bb4041d10 0x7f7bb4044100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bd01097f0 0x7f7bd0109070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 --2- 172.21.15.90:0/3095416481 >> [v2:172.21.15.90:6800/4260623570,v1:172.21.15.90:6801/4260623570] conn(0x7f7bb403e420 0x7f7bb40408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 >> 172.21.15.90:0/3095416481 conn(0x7f7bd0103280 msgr2=0x7f7bd0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 shutdown_connections 2024-06-02T15:29:24.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:24.334+0000 7f7bd5000700 1 -- 172.21.15.90:0/3095416481 wait complete. 2024-06-02T15:29:24.998 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-06-02T15:29:24.998 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:24] ENGINE Bus STARTING 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:24] ENGINE Serving on https://172.21.15.90:7150 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:24] ENGINE Serving on http://172.21.15.90:8765 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:24] ENGINE Bus STARTED 2024-06-02T15:29:25.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:25 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:26.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:26 smithi090 ceph-mon[31595]: mgrmap e8: x(active, since 2s) 2024-06-02T15:29:26.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:26 smithi090 ceph-mon[31595]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:26.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:26 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:26.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:26 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:26.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.603+0000 7fab26a00700 1 Processor -- start 2024-06-02T15:29:26.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.603+0000 7fab26a00700 1 -- start start 2024-06-02T15:29:26.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.604+0000 7fab26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:26.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.604+0000 7fab26a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab20109760 con 0x7fab20108d70 2024-06-02T15:29:26.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.604+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:26.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.604+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42012/0 (socket says 172.21.15.90:42012) 2024-06-02T15:29:26.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.604+0000 7fab1fe00700 1 -- 172.21.15.90:0/2238356388 learned_addr learned my addr 172.21.15.90:0/2238356388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:26.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.605+0000 7fab1fe00700 1 -- 172.21.15.90:0/2238356388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab20109f70 con 0x7fab20108d70 2024-06-02T15:29:26.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.605+0000 7fab1fe00700 1 --2- 172.21.15.90:0/2238356388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab10009e80 tx=0x7fab1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84d310f666e7aee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:26.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab1f400700 1 -- 172.21.15.90:0/2238356388 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10017070 con 0x7fab20108d70 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab1f400700 1 -- 172.21.15.90:0/2238356388 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fab1000b8c0 con 0x7fab20108d70 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab1f400700 1 -- 172.21.15.90:0/2238356388 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10004490 con 0x7fab20108d70 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab26a00700 1 -- 172.21.15.90:0/2238356388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 msgr2=0x7fab20109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab26a00700 1 --2- 172.21.15.90:0/2238356388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab10009e80 tx=0x7fab1000b330 comp rx=0 tx=0).stop 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab26a00700 1 -- 172.21.15.90:0/2238356388 shutdown_connections 2024-06-02T15:29:26.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab26a00700 1 --2- 172.21.15.90:0/2238356388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab20109190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:26.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.606+0000 7fab26a00700 1 -- 172.21.15.90:0/2238356388 >> 172.21.15.90:0/2238356388 conn(0x7fab201043b0 msgr2=0x7fab201067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:26.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.607+0000 7fab26a00700 1 -- 172.21.15.90:0/2238356388 shutdown_connections 2024-06-02T15:29:26.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.607+0000 7fab26a00700 1 -- 172.21.15.90:0/2238356388 wait complete. 2024-06-02T15:29:26.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.608+0000 7fab26a00700 1 Processor -- start 2024-06-02T15:29:26.578 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.608+0000 7fab26a00700 1 -- start start 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.608+0000 7fab26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.608+0000 7fab26a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab2010d320 con 0x7fab20108d70 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.608+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42016/0 (socket says 172.21.15.90:42016) 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1fe00700 1 -- 172.21.15.90:0/2498761077 learned_addr learned my addr 172.21.15.90:0/2498761077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:26.579 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1fe00700 1 -- 172.21.15.90:0/2498761077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2007f090 con 0x7fab20108d70 2024-06-02T15:29:26.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1fe00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b7b0 tx=0x7fab1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:26.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10020020 con 0x7fab20108d70 2024-06-02T15:29:26.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fab10012430 con 0x7fab20108d70 2024-06-02T15:29:26.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab10017040 con 0x7fab20108d70 2024-06-02T15:29:26.580 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.609+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2007d280 con 0x7fab20108d70 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.610+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2007d7c0 con 0x7fab20108d70 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.611+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab20064ba0 con 0x7fab20108d70 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.611+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fab10023020 con 0x7fab20108d70 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.612+0000 7fab1d600700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 0x7fab040408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.612+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab1000f050 con 0x7fab20108d70 2024-06-02T15:29:26.581 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.612+0000 7fab17e00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 0x7fab040408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.613+0000 7fab17e00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 0x7fab040408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.617+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab1002a020 con 0x7fab20108d70 2024-06-02T15:29:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.868+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fab201967d0 con 0x7fab0403e420 2024-06-02T15:29:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.882+0000 7fab1d600700 1 -- 172.21.15.90:0/2498761077 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab201967d0 con 0x7fab0403e420 2024-06-02T15:29:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 msgr2=0x7fab040408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 0x7fab040408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 msgr2=0x7fab2007eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab1000b7b0 tx=0x7fab1000be20 comp rx=0 tx=0).stop 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 shutdown_connections 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fab0403e420 0x7fab040408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:26.583 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 --2- 172.21.15.90:0/2498761077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab20108d70 0x7fab2007eb50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:26.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 >> 172.21.15.90:0/2498761077 conn(0x7fab201043b0 msgr2=0x7fab20104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:26.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 shutdown_connections 2024-06-02T15:29:26.584 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:25.886+0000 7fab26a00700 1 -- 172.21.15.90:0/2498761077 wait complete. 2024-06-02T15:29:28.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-02T15:29:28.079 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.144+0000 7f4b8cc00700 1 Processor -- start 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.144+0000 7f4b8cc00700 1 -- start start 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.145+0000 7f4b8cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.145+0000 7f4b8cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88109760 con 0x7f4b88108d70 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.145+0000 7f4b85e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.145+0000 7f4b85e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42018/0 (socket says 172.21.15.90:42018) 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.145+0000 7f4b85e00700 1 -- 172.21.15.90:0/118353525 learned_addr learned my addr 172.21.15.90:0/118353525 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.146+0000 7f4b85e00700 1 -- 172.21.15.90:0/118353525 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88109f70 con 0x7f4b88108d70 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.146+0000 7f4b85e00700 1 --2- 172.21.15.90:0/118353525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4b780098a0 tx=0x7f4b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa43cdc489e8db25 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.147+0000 7f4b84a00700 1 -- 172.21.15.90:0/118353525 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78004160 con 0x7f4b88108d70 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.147+0000 7f4b84a00700 1 -- 172.21.15.90:0/118353525 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b78004300 con 0x7f4b88108d70 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.147+0000 7f4b84a00700 1 -- 172.21.15.90:0/118353525 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78011af0 con 0x7f4b88108d70 2024-06-02T15:29:28.080 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.147+0000 7f4b8cc00700 1 -- 172.21.15.90:0/118353525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 msgr2=0x7f4b88109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.147+0000 7f4b8cc00700 1 --2- 172.21.15.90:0/118353525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4b780098a0 tx=0x7f4b78009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.148+0000 7f4b8cc00700 1 -- 172.21.15.90:0/118353525 shutdown_connections 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.148+0000 7f4b8cc00700 1 --2- 172.21.15.90:0/118353525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b88109190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.148+0000 7f4b8cc00700 1 -- 172.21.15.90:0/118353525 >> 172.21.15.90:0/118353525 conn(0x7f4b881043b0 msgr2=0x7f4b881067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.148+0000 7f4b8cc00700 1 -- 172.21.15.90:0/118353525 shutdown_connections 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.148+0000 7f4b8cc00700 1 -- 172.21.15.90:0/118353525 wait complete. 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.149+0000 7f4b8cc00700 1 Processor -- start 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.149+0000 7f4b8cc00700 1 -- start start 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.149+0000 7f4b8cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.149+0000 7f4b8cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8810d320 con 0x7f4b88108d70 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.149+0000 7f4b85e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b85e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42026/0 (socket says 172.21.15.90:42026) 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b85e00700 1 -- 172.21.15.90:0/1344900335 learned_addr learned my addr 172.21.15.90:0/1344900335 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:28.081 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b85e00700 1 -- 172.21.15.90:0/1344900335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819f710 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b85e00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000c00 tx=0x7f4b78011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78003b20 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b7801a8c0 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.150+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b78024940 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.151+0000 7f4b8cc00700 1 -- 172.21.15.90:0/1344900335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8819f9a0 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.151+0000 7f4b8cc00700 1 -- 172.21.15.90:0/1344900335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a2690 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.152+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4b78022020 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.152+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b60005330 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.153+0000 7f4b7ea00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 0x7f4b6c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.153+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4b78053860 con 0x7f4b88108d70 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.153+0000 7f4b85400700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 0x7f4b6c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.154+0000 7f4b85400700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 0x7f4b6c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b70006fd0 tx=0x7f4b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:28.082 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.163+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b7801ab20 con 0x7f4b88108d70 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.395+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4b60000bc0 con 0x7f4b6c03e470 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.396+0000 7f4b7ea00700 1 -- 172.21.15.90:0/1344900335 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4b60000bc0 con 0x7f4b6c03e470 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.400+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 msgr2=0x7f4b6c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.400+0000 7f4b77e00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 0x7f4b6c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b70006fd0 tx=0x7f4b70006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.400+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 msgr2=0x7f4b8819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.400+0000 7f4b77e00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b78000c00 tx=0x7f4b78011b70 comp rx=0 tx=0).stop 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 shutdown_connections 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f4b6c03e470 0x7f4b6c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 --2- 172.21.15.90:0/1344900335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b88108d70 0x7f4b8819f1d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 >> 172.21.15.90:0/1344900335 conn(0x7f4b881043b0 msgr2=0x7f4b88105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 shutdown_connections 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:27.401+0000 7f4b77e00700 1 -- 172.21.15.90:0/1344900335 wait complete. 2024-06-02T15:29:28.083 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-06-02T15:29:28.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:28 smithi090 ceph-mon[31595]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:29.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: Generating public/private rsa key pair. 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: Your identification has been saved in /tmp/tmpnjaxgzul/key. 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: Your public key has been saved in /tmp/tmpnjaxgzul/key.pub. 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: The key fingerprint is: 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: SHA256:KN5j49wrIJQKP2/LVfWmX8Rme+MEk04Oa8C9P0auD1I ceph-935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: The key's randomart image is: 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: +---[RSA 3072]----+ 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | . . | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: |. o ..... .. | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: |.+ . . So Eo== | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: |. +..o . ooB=o. | 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | +..* ..=oooo.| 2024-06-02T15:29:29.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | .o=.+ o.o=o..| 2024-06-02T15:29:29.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: | .o.o.o. .=o.. | 2024-06-02T15:29:29.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: +----[SHA256]-----+ 2024-06-02T15:29:29.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f4822a00700 1 Processor -- start 2024-06-02T15:29:29.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f4822a00700 1 -- start start 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f4822a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f4822a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f481c10b870 con 0x7f481c10aef0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f481be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f481be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42042/0 (socket says 172.21.15.90:42042) 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.694+0000 7f481be00700 1 -- 172.21.15.90:0/1564192475 learned_addr learned my addr 172.21.15.90:0/1564192475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.695+0000 7f481be00700 1 -- 172.21.15.90:0/1564192475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c10c0d0 con 0x7f481c10aef0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.695+0000 7f481be00700 1 --2- 172.21.15.90:0/1564192475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9406f7785e7e3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.695+0000 7f481aa00700 1 -- 172.21.15.90:0/1564192475 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c004160 con 0x7f481c10aef0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.695+0000 7f481aa00700 1 -- 172.21.15.90:0/1564192475 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f480c004300 con 0x7f481c10aef0 2024-06-02T15:29:29.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.695+0000 7f481aa00700 1 -- 172.21.15.90:0/1564192475 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c011af0 con 0x7f481c10aef0 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 -- 172.21.15.90:0/1564192475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 msgr2=0x7f481c10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 --2- 172.21.15.90:0/1564192475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f480c0098a0 tx=0x7f480c009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 -- 172.21.15.90:0/1564192475 shutdown_connections 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 --2- 172.21.15.90:0/1564192475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c10b330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 -- 172.21.15.90:0/1564192475 >> 172.21.15.90:0/1564192475 conn(0x7f481c07e310 msgr2=0x7f481c07e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 -- 172.21.15.90:0/1564192475 shutdown_connections 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.696+0000 7f4822a00700 1 -- 172.21.15.90:0/1564192475 wait complete. 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f4822a00700 1 Processor -- start 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f4822a00700 1 -- start start 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f4822a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f4822a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f481c10cf60 con 0x7f481c10aef0 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f481be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:29.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f481be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42054/0 (socket says 172.21.15.90:42054) 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.698+0000 7f481be00700 1 -- 172.21.15.90:0/330468388 learned_addr learned my addr 172.21.15.90:0/330468388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.699+0000 7f481be00700 1 -- 172.21.15.90:0/330468388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c19f700 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.699+0000 7f481be00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f480c009fb0 tx=0x7f480c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.700+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c0036a0 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.700+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f480c01a430 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.700+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c19f990 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.700+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f480c019550 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.700+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a2680 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.702+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c064ba0 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.702+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f480c01a8e0 con 0x7f481c10aef0 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.702+0000 7f4818c00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 0x7f4800040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:29.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.703+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f480c053810 con 0x7f481c10aef0 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.703+0000 7f481b400700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 0x7f4800040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.703+0000 7f481b400700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 0x7f4800040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.708+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f480c00b3e0 con 0x7f481c10aef0 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:28.941+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f481c02e5d0 con 0x7f480003e470 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.071+0000 7f4818c00700 1 -- 172.21.15.90:0/330468388 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f481c02e5d0 con 0x7f480003e470 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.074+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 msgr2=0x7f4800040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.074+0000 7f4822a00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 0x7f4800040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4808006fd0 tx=0x7f4808006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.074+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 msgr2=0x7f481c19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.074+0000 7f4822a00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f480c009fb0 tx=0x7f480c011940 comp rx=0 tx=0).stop 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 shutdown_connections 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f480003e470 0x7f4800040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 --2- 172.21.15.90:0/330468388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f481c10aef0 0x7f481c19f1c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:29.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 >> 172.21.15.90:0/330468388 conn(0x7f481c07e310 msgr2=0x7f481c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:29.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 shutdown_connections 2024-06-02T15:29:29.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:29.075+0000 7f4822a00700 1 -- 172.21.15.90:0/330468388 wait complete. 2024-06-02T15:29:30.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:30 smithi090 ceph-mon[31595]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:30.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:30 smithi090 ceph-mon[31595]: Generating ssh key... 2024-06-02T15:29:30.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:30 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:30.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:30 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:31.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:31 smithi090 ceph-mon[31595]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:31.379 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:29:31.381 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.442+0000 7faebf000700 1 Processor -- start 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faebf000700 1 -- start start 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faebf000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faebf000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faeb807cdb0 con 0x7faeb81098c0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faeb7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faeb7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42058/0 (socket says 172.21.15.90:42058) 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.443+0000 7faeb7e00700 1 -- 172.21.15.90:0/1676181424 learned_addr learned my addr 172.21.15.90:0/1676181424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.444+0000 7faeb7e00700 1 -- 172.21.15.90:0/1676181424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb810c1f0 con 0x7faeb81098c0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.444+0000 7faeb7e00700 1 --2- 172.21.15.90:0/1676181424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faea80098a0 tx=0x7faea8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1b9650fde7b48e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.444+0000 7faeb6a00700 1 -- 172.21.15.90:0/1676181424 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea8004160 con 0x7faeb81098c0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.444+0000 7faeb6a00700 1 -- 172.21.15.90:0/1676181424 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faea8004300 con 0x7faeb81098c0 2024-06-02T15:29:31.382 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.444+0000 7faeb6a00700 1 -- 172.21.15.90:0/1676181424 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea8011af0 con 0x7faeb81098c0 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 -- 172.21.15.90:0/1676181424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 msgr2=0x7faeb810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 --2- 172.21.15.90:0/1676181424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faea80098a0 tx=0x7faea8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 -- 172.21.15.90:0/1676181424 shutdown_connections 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 --2- 172.21.15.90:0/1676181424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb810bcb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 -- 172.21.15.90:0/1676181424 >> 172.21.15.90:0/1676181424 conn(0x7faeb8103210 msgr2=0x7faeb8105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 -- 172.21.15.90:0/1676181424 shutdown_connections 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.445+0000 7faebf000700 1 -- 172.21.15.90:0/1676181424 wait complete. 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.446+0000 7faebf000700 1 Processor -- start 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faebf000700 1 -- start start 2024-06-02T15:29:31.383 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faebf000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faebf000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faeb810d0f0 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faeb7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faeb7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42068/0 (socket says 172.21.15.90:42068) 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faeb7e00700 1 -- 172.21.15.90:0/2592731225 learned_addr learned my addr 172.21.15.90:0/2592731225 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.447+0000 7faeb7e00700 1 -- 172.21.15.90:0/2592731225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb819f700 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faeb7e00700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea8011ac0 tx=0x7faea8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea8003970 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faea8003b10 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faea8024610 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb819f990 con 0x7faeb81098c0 2024-06-02T15:29:31.387 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.448+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb81a2680 con 0x7faeb81098c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.449+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7faea801a9e0 con 0x7faeb81098c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.449+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb8064ba0 con 0x7faeb81098c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.450+0000 7faeb4c00700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 0x7fae9c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.450+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faea8053910 con 0x7faeb81098c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.450+0000 7faeb7400700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 0x7fae9c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.451+0000 7faeb7400700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 0x7fae9c040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faea4006fd0 tx=0x7faea4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.455+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faea8053260 con 0x7faeb81098c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.689+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7faeb802e5d0 con 0x7fae9c03e4c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.691+0000 7faeb4c00700 1 -- 172.21.15.90:0/2592731225 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7faeb802e5d0 con 0x7fae9c03e4c0 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 msgr2=0x7fae9c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 0x7fae9c040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faea4006fd0 tx=0x7faea4006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:31.388 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 msgr2=0x7faeb819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faea8011ac0 tx=0x7faea8011ba0 comp rx=0 tx=0).stop 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 shutdown_connections 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fae9c03e4c0 0x7fae9c040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 --2- 172.21.15.90:0/2592731225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb81098c0 0x7faeb819f1c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 >> 172.21.15.90:0/2592731225 conn(0x7faeb8103210 msgr2=0x7faeb8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.694+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 shutdown_connections 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:30.695+0000 7faebf000700 1 -- 172.21.15.90:0/2592731225 wait complete. 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-06-02T15:29:31.389 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-06-02T15:29:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:33 smithi090 ceph-mon[31595]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:34.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:34 smithi090 ceph-mon[31595]: Deploying cephadm binary to smithi090 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.964+0000 7f2e95c00700 1 Processor -- start 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.964+0000 7f2e95c00700 1 -- start start 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.965+0000 7f2e95c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.965+0000 7f2e95c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90109760 con 0x7f2e90108d70 2024-06-02T15:29:38.847 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.965+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.965+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47158/0 (socket says 172.21.15.90:47158) 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.965+0000 7f2e8ea00700 1 -- 172.21.15.90:0/1756806209 learned_addr learned my addr 172.21.15.90:0/1756806209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.966+0000 7f2e8ea00700 1 -- 172.21.15.90:0/1756806209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e90109f70 con 0x7f2e90108d70 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.966+0000 7f2e8ea00700 1 --2- 172.21.15.90:0/1756806209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e800098a0 tx=0x7f2e80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6750a3ca2121689c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.966+0000 7f2e8d600700 1 -- 172.21.15.90:0/1756806209 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80004160 con 0x7f2e90108d70 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.966+0000 7f2e8d600700 1 -- 172.21.15.90:0/1756806209 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e80004300 con 0x7f2e90108d70 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.966+0000 7f2e8d600700 1 -- 172.21.15.90:0/1756806209 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80011af0 con 0x7f2e90108d70 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.967+0000 7f2e95c00700 1 -- 172.21.15.90:0/1756806209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 msgr2=0x7f2e90109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.967+0000 7f2e95c00700 1 --2- 172.21.15.90:0/1756806209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e800098a0 tx=0x7f2e80009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.967+0000 7f2e95c00700 1 -- 172.21.15.90:0/1756806209 shutdown_connections 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.967+0000 7f2e95c00700 1 --2- 172.21.15.90:0/1756806209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e90109190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.967+0000 7f2e95c00700 1 -- 172.21.15.90:0/1756806209 >> 172.21.15.90:0/1756806209 conn(0x7f2e901043b0 msgr2=0x7f2e901067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:38.848 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.968+0000 7f2e95c00700 1 -- 172.21.15.90:0/1756806209 shutdown_connections 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.968+0000 7f2e95c00700 1 -- 172.21.15.90:0/1756806209 wait complete. 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.969+0000 7f2e95c00700 1 Processor -- start 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.969+0000 7f2e95c00700 1 -- start start 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.969+0000 7f2e95c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.969+0000 7f2e95c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9010d320 con 0x7f2e90108d70 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.970+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.970+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47164/0 (socket says 172.21.15.90:47164) 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.970+0000 7f2e8ea00700 1 -- 172.21.15.90:0/63622771 learned_addr learned my addr 172.21.15.90:0/63622771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.970+0000 7f2e8ea00700 1 -- 172.21.15.90:0/63622771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9019d510 con 0x7f2e90108d70 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.970+0000 7f2e8ea00700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e80000c00 tx=0x7f2e80011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.971+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80003b20 con 0x7f2e90108d70 2024-06-02T15:29:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.971+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e8001a8c0 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.971+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9019d7a0 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.971+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80024940 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.971+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e901a0490 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.972+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2e80022020 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.972+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e90064ba0 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.973+0000 7f2e87400700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 0x7f2e74040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.973+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2e80053740 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.973+0000 7f2e8e000700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 0x7f2e74040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.974+0000 7f2e8e000700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 0x7f2e74040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006fd0 tx=0x7f2e7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:31.980+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e8001aa70 con 0x7f2e90108d70 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:32.214+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) -- 0x7f2e9002e5d0 con 0x7f2e7403e4c0 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.149+0000 7f2e87400700 1 -- 172.21.15.90:0/63622771 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2e9002e5d0 con 0x7f2e7403e4c0 2024-06-02T15:29:38.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 msgr2=0x7f2e74040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 0x7f2e74040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c006fd0 tx=0x7f2e7c006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 msgr2=0x7f2e9019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e80000c00 tx=0x7f2e80011b70 comp rx=0 tx=0).stop 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 shutdown_connections 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f2e7403e4c0 0x7f2e74040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 --2- 172.21.15.90:0/63622771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e90108d70 0x7f2e9019cfd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.153+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 >> 172.21.15.90:0/63622771 conn(0x7f2e901043b0 msgr2=0x7f2e90105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.154+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 shutdown_connections 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:38.154+0000 7f2e95c00700 1 -- 172.21.15.90:0/63622771 wait complete. 2024-06-02T15:29:38.851 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-06-02T15:29:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:39 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:39 smithi090 ceph-mon[31595]: Added host smithi090 2024-06-02T15:29:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:39 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:41.004 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:40 smithi090 ceph-mon[31595]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:41.004 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:40 smithi090 ceph-mon[31595]: Saving service mon spec with placement count:5 2024-06-02T15:29:41.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:40 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:41.017 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.489+0000 7fa985600700 1 Processor -- start 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.489+0000 7fa985600700 1 -- start start 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.489+0000 7fa985600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.489+0000 7fa985600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa980109550 con 0x7fa980108b60 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.490+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.490+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47176/0 (socket says 172.21.15.90:47176) 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.490+0000 7fa97fe00700 1 -- 172.21.15.90:0/1326202323 learned_addr learned my addr 172.21.15.90:0/1326202323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.490+0000 7fa97fe00700 1 -- 172.21.15.90:0/1326202323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa980109d60 con 0x7fa980108b60 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.491+0000 7fa97fe00700 1 --2- 172.21.15.90:0/1326202323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098a0 tx=0x7fa970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b03e8254f0ec40 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:41.018 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.491+0000 7fa97ea00700 1 -- 172.21.15.90:0/1326202323 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa970004160 con 0x7fa980108b60 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.491+0000 7fa97ea00700 1 -- 172.21.15.90:0/1326202323 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa970004300 con 0x7fa980108b60 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.491+0000 7fa97ea00700 1 -- 172.21.15.90:0/1326202323 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa970011af0 con 0x7fa980108b60 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 -- 172.21.15.90:0/1326202323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 msgr2=0x7fa980108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 --2- 172.21.15.90:0/1326202323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098a0 tx=0x7fa970009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 -- 172.21.15.90:0/1326202323 shutdown_connections 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 --2- 172.21.15.90:0/1326202323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa980108f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:41.019 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 -- 172.21.15.90:0/1326202323 >> 172.21.15.90:0/1326202323 conn(0x7fa980104160 msgr2=0x7fa980106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 -- 172.21.15.90:0/1326202323 shutdown_connections 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.492+0000 7fa985600700 1 -- 172.21.15.90:0/1326202323 wait complete. 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.493+0000 7fa985600700 1 Processor -- start 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.493+0000 7fa985600700 1 -- start start 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.493+0000 7fa985600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.493+0000 7fa985600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa98010d110 con 0x7fa980108b60 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.494+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.494+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47188/0 (socket says 172.21.15.90:47188) 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.494+0000 7fa97fe00700 1 -- 172.21.15.90:0/3743809910 learned_addr learned my addr 172.21.15.90:0/3743809910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.494+0000 7fa97fe00700 1 -- 172.21.15.90:0/3743809910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98019f280 con 0x7fa980108b60 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.494+0000 7fa97fe00700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9700117b0 tx=0x7fa970011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.495+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa970003af0 con 0x7fa980108b60 2024-06-02T15:29:41.020 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.495+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa970020070 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.495+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa970019630 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.495+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98019f510 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.495+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9801a2200 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.496+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa97001a9c0 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.496+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa980064ba0 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.497+0000 7fa97cc00700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 0x7fa964040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.497+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa970053a60 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.497+0000 7fa97f400700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 0x7fa964040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.498+0000 7fa97f400700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 0x7fa964040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.503+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa97000bdd0 con 0x7fa980108b60 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.758+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa980105310 con 0x7fa96403e4c0 2024-06-02T15:29:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.765+0000 7fa97cc00700 1 -- 172.21.15.90:0/3743809910 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa980105310 con 0x7fa96403e4c0 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.768+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 msgr2=0x7fa964040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.768+0000 7fa985600700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 0x7fa964040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 msgr2=0x7fa98019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9700117b0 tx=0x7fa970011b70 comp rx=0 tx=0).stop 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 shutdown_connections 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fa96403e4c0 0x7fa964040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 --2- 172.21.15.90:0/3743809910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa980108b60 0x7fa98019ed40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 >> 172.21.15.90:0/3743809910 conn(0x7fa980104160 msgr2=0x7fa980104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 shutdown_connections 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:39.769+0000 7fa985600700 1 -- 172.21.15.90:0/3743809910 wait complete. 2024-06-02T15:29:41.022 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-06-02T15:29:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:42 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:42 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:42 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:42.918 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.016+0000 7f51bd400700 1 Processor -- start 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.016+0000 7f51bd400700 1 -- start start 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.016+0000 7f51bd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.016+0000 7f51bd400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f51b807ce90 con 0x7f51b81097f0 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.017+0000 7f51b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.017+0000 7f51b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48344/0 (socket says 172.21.15.90:48344) 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.017+0000 7f51b6800700 1 -- 172.21.15.90:0/1985877557 learned_addr learned my addr 172.21.15.90:0/1985877557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.017+0000 7f51b6800700 1 -- 172.21.15.90:0/1985877557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b810c120 con 0x7f51b81097f0 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.018+0000 7f51b6800700 1 --2- 172.21.15.90:0/1985877557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51a80098a0 tx=0x7f51a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f185d2194b4ca62 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.018+0000 7f51b5400700 1 -- 172.21.15.90:0/1985877557 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8004160 con 0x7f51b81097f0 2024-06-02T15:29:42.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.018+0000 7f51b5400700 1 -- 172.21.15.90:0/1985877557 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a8004300 con 0x7f51b81097f0 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.018+0000 7f51b5400700 1 -- 172.21.15.90:0/1985877557 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8011a20 con 0x7f51b81097f0 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 -- 172.21.15.90:0/1985877557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 msgr2=0x7f51b810bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 --2- 172.21.15.90:0/1985877557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51a80098a0 tx=0x7f51a8009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 -- 172.21.15.90:0/1985877557 shutdown_connections 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 --2- 172.21.15.90:0/1985877557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b810bbe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 -- 172.21.15.90:0/1985877557 >> 172.21.15.90:0/1985877557 conn(0x7f51b8103280 msgr2=0x7f51b81056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 -- 172.21.15.90:0/1985877557 shutdown_connections 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.019+0000 7f51bd400700 1 -- 172.21.15.90:0/1985877557 wait complete. 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.020+0000 7f51bd400700 1 Processor -- start 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.020+0000 7f51bd400700 1 -- start start 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.021+0000 7f51bd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.021+0000 7f51bd400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f51b810d130 con 0x7f51b81097f0 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.021+0000 7f51b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:42.920 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.021+0000 7f51b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48360/0 (socket says 172.21.15.90:48360) 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.021+0000 7f51b6800700 1 -- 172.21.15.90:0/3834675274 learned_addr learned my addr 172.21.15.90:0/3834675274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51b6800700 1 -- 172.21.15.90:0/3834675274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b819d480 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51b6800700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000c00 tx=0x7f51a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8003aa0 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a801b8f0 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51a8023870 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.022+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b819d710 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.023+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81a0400 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.024+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b8064ba0 con 0x7f51b81097f0 2024-06-02T15:29:42.921 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.025+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f51a801b440 con 0x7f51b81097f0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.025+0000 7f51af400700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 0x7f519c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.025+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51a8053ed0 con 0x7f51b81097f0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.025+0000 7f51b5e00700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 0x7f519c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.026+0000 7f51b5e00700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 0x7f519c040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006fd0 tx=0x7f51a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.031+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51a801f030 con 0x7f51b81097f0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.287+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f51b8063980 con 0x7f519c03e4c0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.294+0000 7f51af400700 1 -- 172.21.15.90:0/3834675274 <== mgr.14120 v2:172.21.15.90:6800/1560565398 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f51b8063980 con 0x7f519c03e4c0 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.296+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 msgr2=0x7f519c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:42.922 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.296+0000 7f51bd400700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 0x7f519c040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006fd0 tx=0x7f51a4006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.296+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 msgr2=0x7f51b819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.296+0000 7f51bd400700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000c00 tx=0x7f51a8011aa0 comp rx=0 tx=0).stop 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 shutdown_connections 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f519c03e4c0 0x7f519c040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 --2- 172.21.15.90:0/3834675274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51b81097f0 0x7f51b819cf40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 >> 172.21.15.90:0/3834675274 conn(0x7f51b8103280 msgr2=0x7f51b8103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 shutdown_connections 2024-06-02T15:29:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:42.297+0000 7f51bd400700 1 -- 172.21.15.90:0/3834675274 wait complete. 2024-06-02T15:29:43.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:43 smithi090 ceph-mon[31595]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:29:43.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:43 smithi090 ceph-mon[31595]: Saving service mgr spec with placement count:2 2024-06-02T15:29:44.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:44 smithi090 ceph-mon[31595]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:29:44.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3193296367' entity='client.admin' 2024-06-02T15:29:44.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.449+0000 7fec89000700 1 Processor -- start 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.449+0000 7fec89000700 1 -- start start 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.450+0000 7fec89000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.450+0000 7fec89000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fec84109730 con 0x7fec84108d40 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.450+0000 7fec81e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.450+0000 7fec81e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48374/0 (socket says 172.21.15.90:48374) 2024-06-02T15:29:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.450+0000 7fec81e00700 1 -- 172.21.15.90:0/1395756041 learned_addr learned my addr 172.21.15.90:0/1395756041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.451+0000 7fec81e00700 1 -- 172.21.15.90:0/1395756041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec84109f40 con 0x7fec84108d40 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.451+0000 7fec81e00700 1 --2- 172.21.15.90:0/1395756041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec740098a0 tx=0x7fec74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edc0012dfa41c93 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec80a00700 1 -- 172.21.15.90:0/1395756041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec74004160 con 0x7fec84108d40 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec80a00700 1 -- 172.21.15.90:0/1395756041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec74004300 con 0x7fec84108d40 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec80a00700 1 -- 172.21.15.90:0/1395756041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec74011a20 con 0x7fec84108d40 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec89000700 1 -- 172.21.15.90:0/1395756041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 msgr2=0x7fec84109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec89000700 1 --2- 172.21.15.90:0/1395756041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fec740098a0 tx=0x7fec74009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec89000700 1 -- 172.21.15.90:0/1395756041 shutdown_connections 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec89000700 1 --2- 172.21.15.90:0/1395756041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec84109160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.452+0000 7fec89000700 1 -- 172.21.15.90:0/1395756041 >> 172.21.15.90:0/1395756041 conn(0x7fec841042e0 msgr2=0x7fec84106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.453+0000 7fec89000700 1 -- 172.21.15.90:0/1395756041 shutdown_connections 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.453+0000 7fec89000700 1 -- 172.21.15.90:0/1395756041 wait complete. 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec89000700 1 Processor -- start 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec89000700 1 -- start start 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec89000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec89000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410d2f0 con 0x7fec84108d40 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec81e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec81e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48388/0 (socket says 172.21.15.90:48388) 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.454+0000 7fec81e00700 1 -- 172.21.15.90:0/3193296367 learned_addr learned my addr 172.21.15.90:0/3193296367 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec81e00700 1 -- 172.21.15.90:0/3193296367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8419d4c0 con 0x7fec84108d40 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec81e00700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec740119f0 tx=0x7fec74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec740039e0 con 0x7fec84108d40 2024-06-02T15:29:44.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7401b8f0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec740237a0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.455+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8419d750 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.456+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8407d0c0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.457+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fec7401b440 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.457+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec84064ba0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.457+0000 7fec7aa00700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 0x7fec68044d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.457+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fec74053ed0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.458+0000 7fec81400700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 0x7fec68044d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.458+0000 7fec81400700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 0x7fec68044d40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec70006fd0 tx=0x7fec70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.467+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec7401f030 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.705+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fec841965c0 con 0x7fec84108d40 2024-06-02T15:29:44.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.711+0000 7fec7aa00700 1 -- 172.21.15.90:0/3193296367 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fec74016260 con 0x7fec84108d40 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 msgr2=0x7fec68044d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 0x7fec68044d40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec70006fd0 tx=0x7fec70006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 msgr2=0x7fec8419cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec740119f0 tx=0x7fec74011ad0 comp rx=0 tx=0).stop 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 shutdown_connections 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fec68042880 0x7fec68044d40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:44.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 --2- 172.21.15.90:0/3193296367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec84108d40 0x7fec8419cf80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:44.939 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.714+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 >> 172.21.15.90:0/3193296367 conn(0x7fec841042e0 msgr2=0x7fec84104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:44.939 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.715+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 shutdown_connections 2024-06-02T15:29:44.939 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:43.715+0000 7fec89000700 1 -- 172.21.15.90:0/3193296367 wait complete. 2024-06-02T15:29:46.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:46 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:46.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:46 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1596806690' entity='client.admin' 2024-06-02T15:29:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.582+0000 7f82e1c00700 1 Processor -- start 2024-06-02T15:29:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.582+0000 7f82e1c00700 1 -- start start 2024-06-02T15:29:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.582+0000 7f82e1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.582+0000 7f82e1c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc07d8b0 con 0x7f82dc07ee80 2024-06-02T15:29:46.561 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.583+0000 7f82daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.583+0000 7f82daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48396/0 (socket says 172.21.15.90:48396) 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.583+0000 7f82daa00700 1 -- 172.21.15.90:0/3175024496 learned_addr learned my addr 172.21.15.90:0/3175024496 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.583+0000 7f82daa00700 1 -- 172.21.15.90:0/3175024496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc07da30 con 0x7f82dc07ee80 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.584+0000 7f82daa00700 1 --2- 172.21.15.90:0/3175024496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098a0 tx=0x7f82cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f946323efa2733a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.584+0000 7f82d9600700 1 -- 172.21.15.90:0/3175024496 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82cc004160 con 0x7f82dc07ee80 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.584+0000 7f82d9600700 1 -- 172.21.15.90:0/3175024496 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc004300 con 0x7f82dc07ee80 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.584+0000 7f82d9600700 1 -- 172.21.15.90:0/3175024496 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82cc011a20 con 0x7f82dc07ee80 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 -- 172.21.15.90:0/3175024496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 msgr2=0x7f82dc07d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 --2- 172.21.15.90:0/3175024496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0098a0 tx=0x7f82cc009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 -- 172.21.15.90:0/3175024496 shutdown_connections 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 --2- 172.21.15.90:0/3175024496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc07d2e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 -- 172.21.15.90:0/3175024496 >> 172.21.15.90:0/3175024496 conn(0x7f82dc1043b0 msgr2=0x7f82dc1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 -- 172.21.15.90:0/3175024496 shutdown_connections 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.585+0000 7f82e1c00700 1 -- 172.21.15.90:0/3175024496 wait complete. 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.586+0000 7f82e1c00700 1 Processor -- start 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.586+0000 7f82e1c00700 1 -- start start 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.586+0000 7f82e1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.586+0000 7f82e1c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10f370 con 0x7f82dc07ee80 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.587+0000 7f82daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.587+0000 7f82daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48398/0 (socket says 172.21.15.90:48398) 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.587+0000 7f82daa00700 1 -- 172.21.15.90:0/1596806690 learned_addr learned my addr 172.21.15.90:0/1596806690 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.587+0000 7f82daa00700 1 -- 172.21.15.90:0/1596806690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19f670 con 0x7f82dc07ee80 2024-06-02T15:29:46.563 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.587+0000 7f82daa00700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009520 tx=0x7f82cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.588+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82cc0039e0 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.588+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82cc01b8f0 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.588+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82cc019770 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.588+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc19f900 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.588+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a25f0 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.589+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc064ba0 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.590+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f82cc01b440 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.590+0000 7f82d3400700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 0x7f82c0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.590+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82cc053fd0 con 0x7f82dc07ee80 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.590+0000 7f82da000700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 0x7f82c0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.591+0000 7f82da000700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 0x7f82c0040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82c8006fd0 tx=0x7f82c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:46.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.600+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82cc01f030 con 0x7f82dc07ee80 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.838+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f82dc02e560 con 0x7f82dc07ee80 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.846+0000 7f82d3400700 1 -- 172.21.15.90:0/1596806690 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f82cc018d60 con 0x7f82dc07ee80 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 msgr2=0x7f82c0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 0x7f82c0040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f82c8006fd0 tx=0x7f82c8006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 msgr2=0x7f82dc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009520 tx=0x7f82cc011aa0 comp rx=0 tx=0).stop 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 shutdown_connections 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f82c003e470 0x7f82c0040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.848+0000 7f82e1c00700 1 --2- 172.21.15.90:0/1596806690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82dc07ee80 0x7f82dc19f130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.849+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 >> 172.21.15.90:0/1596806690 conn(0x7f82dc1043b0 msgr2=0x7f82dc104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.849+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 shutdown_connections 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:45.849+0000 7f82e1c00700 1 -- 172.21.15.90:0/1596806690 wait complete. 2024-06-02T15:29:46.565 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-06-02T15:29:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:47 smithi090 ceph-mon[31595]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:29:48.244 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-mon[31595]: from='mgr.14120 172.21.15.90:0/258300341' entity='mgr.x' 2024-06-02T15:29:48.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1717438274' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-02T15:29:48.599 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: ignoring --setuser ceph since I am not root 2024-06-02T15:29:48.600 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: ignoring --setgroup ceph since I am not root 2024-06-02T15:29:48.600 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:48.439+0000 7effcbb15200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T15:29:48.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:48.699+0000 7effcbb15200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:48.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:48.830+0000 7effcbb15200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T15:29:49.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:49 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1717438274' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-02T15:29:49.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:49 smithi090 ceph-mon[31595]: mgrmap e9: x(active, since 24s) 2024-06-02T15:29:49.337 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:49.231+0000 7effcbb15200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T15:29:49.343 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.183+0000 7f7f2a000700 1 Processor -- start 2024-06-02T15:29:49.344 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.183+0000 7f7f2a000700 1 -- start start 2024-06-02T15:29:49.344 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.183+0000 7f7f2a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:49.344 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.183+0000 7f7f2a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24109760 con 0x7f7f24108d70 2024-06-02T15:29:49.344 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.184+0000 7f7f23400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:49.345 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.184+0000 7f7f23400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48400/0 (socket says 172.21.15.90:48400) 2024-06-02T15:29:49.345 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.184+0000 7f7f23400700 1 -- 172.21.15.90:0/2683026245 learned_addr learned my addr 172.21.15.90:0/2683026245 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:49.345 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.184+0000 7f7f23400700 1 -- 172.21.15.90:0/2683026245 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f24109f70 con 0x7f7f24108d70 2024-06-02T15:29:49.345 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.185+0000 7f7f23400700 1 --2- 172.21.15.90:0/2683026245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f140098a0 tx=0x7f7f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c40d3355973b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:49.345 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.185+0000 7f7f22000700 1 -- 172.21.15.90:0/2683026245 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f14004160 con 0x7f7f24108d70 2024-06-02T15:29:49.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.185+0000 7f7f22000700 1 -- 172.21.15.90:0/2683026245 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f14004300 con 0x7f7f24108d70 2024-06-02T15:29:49.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.185+0000 7f7f22000700 1 -- 172.21.15.90:0/2683026245 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f14011a20 con 0x7f7f24108d70 2024-06-02T15:29:49.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 -- 172.21.15.90:0/2683026245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 msgr2=0x7f7f24109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:49.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 --2- 172.21.15.90:0/2683026245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f140098a0 tx=0x7f7f14009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:49.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 -- 172.21.15.90:0/2683026245 shutdown_connections 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 --2- 172.21.15.90:0/2683026245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f24109190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 -- 172.21.15.90:0/2683026245 >> 172.21.15.90:0/2683026245 conn(0x7f7f241043b0 msgr2=0x7f7f241067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 -- 172.21.15.90:0/2683026245 shutdown_connections 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.186+0000 7f7f2a000700 1 -- 172.21.15.90:0/2683026245 wait complete. 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.187+0000 7f7f2a000700 1 Processor -- start 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.187+0000 7f7f2a000700 1 -- start start 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f2a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f2a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410d320 con 0x7f7f24108d70 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f23400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f23400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48402/0 (socket says 172.21.15.90:48402) 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f23400700 1 -- 172.21.15.90:0/1717438274 learned_addr learned my addr 172.21.15.90:0/1717438274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:49.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f23400700 1 -- 172.21.15.90:0/1717438274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419f690 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.188+0000 7f7f23400700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009520 tx=0x7f7f14011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.189+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f140039e0 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.189+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1401b8f0 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.189+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2419f920 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.189+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f14019770 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.189+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a2610 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.190+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24064ba0 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.190+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7f14019a20 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.191+0000 7f7f1be00700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 0x7f7f08040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.191+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f14053e40 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.191+0000 7f7f22a00700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 0x7f7f08040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.192+0000 7f7f22a00700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 0x7f7f08040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.197+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f1401f030 con 0x7f7f24108d70 2024-06-02T15:29:49.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:47.501+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7f2402e560 con 0x7f7f24108d70 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.118+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7f1400b3e0 con 0x7f7f24108d70 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.123+0000 7f7f1be00700 1 -- 172.21.15.90:0/1717438274 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7f14024990 con 0x7f7f24108d70 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.124+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 msgr2=0x7f7f08040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.124+0000 7f7f2a000700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 0x7f7f08040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10006e40 comp rx=0 tx=0).stop 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.124+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 msgr2=0x7f7f2419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.124+0000 7f7f2a000700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009520 tx=0x7f7f14011aa0 comp rx=0 tx=0).stop 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 shutdown_connections 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f7f0803e4c0 0x7f7f08040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 --2- 172.21.15.90:0/1717438274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f24108d70 0x7f7f2419f150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 >> 172.21.15.90:0/1717438274 conn(0x7f7f241043b0 msgr2=0x7f7f24104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 shutdown_connections 2024-06-02T15:29:49.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:48.125+0000 7f7f2a000700 1 -- 172.21.15.90:0/1717438274 wait complete. 2024-06-02T15:29:49.623 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:49.350+0000 7effcbb15200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T15:29:51.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:50 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3139069714' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-02T15:29:51.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:51.001+0000 7effcbb15200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T15:29:51.449 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:51.232+0000 7effcbb15200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:51.449 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:51.450+0000 7effcbb15200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.476+0000 7f3d5e800700 1 Processor -- start 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5e800700 1 -- start start 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:51.467 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5e800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5807ce90 con 0x7f3d581095e0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48434/0 (socket says 172.21.15.90:48434) 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.477+0000 7f3d5d400700 1 -- 172.21.15.90:0/3743774224 learned_addr learned my addr 172.21.15.90:0/3743774224 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.478+0000 7f3d5d400700 1 -- 172.21.15.90:0/3743774224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5810bf10 con 0x7f3d581095e0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.478+0000 7f3d5d400700 1 --2- 172.21.15.90:0/3743774224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098a0 tx=0x7f3d4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4373c3d3bcd5644 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.478+0000 7f3d57e00700 1 -- 172.21.15.90:0/3743774224 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d4c004160 con 0x7f3d581095e0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.479+0000 7f3d57e00700 1 -- 172.21.15.90:0/3743774224 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c004300 con 0x7f3d581095e0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.479+0000 7f3d57e00700 1 -- 172.21.15.90:0/3743774224 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d4c011a20 con 0x7f3d581095e0 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.479+0000 7f3d5e800700 1 -- 172.21.15.90:0/3743774224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 msgr2=0x7f3d5810b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.479+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3743774224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098a0 tx=0x7f3d4c009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.480+0000 7f3d5e800700 1 -- 172.21.15.90:0/3743774224 shutdown_connections 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.480+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3743774224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5810b9d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.468 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.480+0000 7f3d5e800700 1 -- 172.21.15.90:0/3743774224 >> 172.21.15.90:0/3743774224 conn(0x7f3d58103010 msgr2=0x7f3d58105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.480+0000 7f3d5e800700 1 -- 172.21.15.90:0/3743774224 shutdown_connections 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.480+0000 7f3d5e800700 1 -- 172.21.15.90:0/3743774224 wait complete. 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.481+0000 7f3d5e800700 1 Processor -- start 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.481+0000 7f3d5e800700 1 -- start start 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.481+0000 7f3d5e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.481+0000 7f3d5e800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5810cf20 con 0x7f3d581095e0 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48440/0 (socket says 172.21.15.90:48440) 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5d400700 1 -- 172.21.15.90:0/3139069714 learned_addr learned my addr 172.21.15.90:0/3139069714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5d400700 1 -- 172.21.15.90:0/3139069714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5819f500 con 0x7f3d581095e0 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5d400700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c004660 tx=0x7f3d4c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d4c003aa0 con 0x7f3d581095e0 2024-06-02T15:29:51.469 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c01b8f0 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d4c023900 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.482+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5819f790 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.483+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d581a2430 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.484+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d58064ba0 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.484+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f3d4c01baa0 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.485+0000 7f3d56000700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 0x7f3d40040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.485+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3d4c053e90 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.485+0000 7f3d5ca00700 1 -- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 msgr2=0x7f3d40040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.485+0000 7f3d5ca00700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 0x7f3d40040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.490+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d4c01f030 con 0x7f3d581095e0 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.686+0000 7f3d5ca00700 1 -- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 msgr2=0x7f3d40040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.686+0000 7f3d5ca00700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 0x7f3d40040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T15:29:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.790+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3d5802e560 con 0x7f3d581095e0 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.791+0000 7f3d56000700 1 -- 172.21.15.90:0/3139069714 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3d4c006d60 con 0x7f3d581095e0 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 msgr2=0x7f3d40040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 0x7f3d40040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 msgr2=0x7f3d5819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c004660 tx=0x7f3d4c011aa0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 shutdown_connections 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7f3d4003e4c0 0x7f3d40040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 --2- 172.21.15.90:0/3139069714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d581095e0 0x7f3d5819efc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 >> 172.21.15.90:0/3139069714 conn(0x7f3d58103010 msgr2=0x7f3d58103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 shutdown_connections 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:50.794+0000 7f3d5e800700 1 -- 172.21.15.90:0/3139069714 wait complete. 2024-06-02T15:29:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-06-02T15:29:51.472 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-06-02T15:29:51.977 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:51.849+0000 7effcbb15200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T15:29:52.234 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:51.979+0000 7effcbb15200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T15:29:52.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:52.235+0000 7effcbb15200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T15:29:52.487 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:52.488+0000 7effcbb15200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:29:53.559 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:53.273+0000 7effcbb15200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T15:29:53.559 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:53.560+0000 7effcbb15200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-06-02T15:29:54.374 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:54.109+0000 7effcbb15200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T15:29:54.384 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:54.236+0000 7effcbb15200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T15:29:55.022 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:54.702+0000 7effcbb15200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T15:29:55.275 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:55.152+0000 7effcbb15200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T15:29:55.276 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:55.276+0000 7effcbb15200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T15:29:55.873 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:55.612+0000 7effcbb15200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T15:29:55.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:55.802+0000 7effcbb15200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T15:29:56.344 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:55.935+0000 7effcbb15200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T15:29:56.344 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:56.344+0000 7effcbb15200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T15:29:56.623 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:56.470+0000 7effcbb15200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T15:29:57.526 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:57.239+0000 7effcbb15200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T15:29:57.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:29:57.527+0000 7effcbb15200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T15:29:57.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: Active manager daemon x restarted 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: Activating manager daemon x 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: osdmap e3: 0 total, 0 up, 0 in 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: mgrmap e10: x(active, starting, since 0.00774234s) 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T15:29:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T15:29:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:57 smithi090 ceph-mon[31595]: Manager daemon x is now available 2024-06-02T15:29:58.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:29:58.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T15:29:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T15:29:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:58 smithi090 ceph-mon[31595]: mgrmap e11: x(active, since 1.01496s) 2024-06-02T15:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.011+0000 7fd776a00700 1 Processor -- start 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.011+0000 7fd776a00700 1 -- start start 2024-06-02T15:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.011+0000 7fd776a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.012+0000 7fd776a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd77007d8f0 con 0x7fd77007eec0 2024-06-02T15:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.012+0000 7fd76fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.012+0000 7fd76fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35304/0 (socket says 172.21.15.90:35304) 2024-06-02T15:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.012+0000 7fd76fe00700 1 -- 172.21.15.90:0/2616641942 learned_addr learned my addr 172.21.15.90:0/2616641942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.012+0000 7fd76fe00700 1 -- 172.21.15.90:0/2616641942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77007da70 con 0x7fd77007eec0 2024-06-02T15:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.013+0000 7fd76fe00700 1 --2- 172.21.15.90:0/2616641942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098a0 tx=0x7fd760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cc932e72741066 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.013+0000 7fd76ea00700 1 -- 172.21.15.90:0/2616641942 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760004160 con 0x7fd77007eec0 2024-06-02T15:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.013+0000 7fd76ea00700 1 -- 172.21.15.90:0/2616641942 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760004300 con 0x7fd77007eec0 2024-06-02T15:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.013+0000 7fd76ea00700 1 -- 172.21.15.90:0/2616641942 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760011a20 con 0x7fd77007eec0 2024-06-02T15:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 -- 172.21.15.90:0/2616641942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 msgr2=0x7fd77007d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 --2- 172.21.15.90:0/2616641942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098a0 tx=0x7fd760009bb0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 -- 172.21.15.90:0/2616641942 shutdown_connections 2024-06-02T15:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 --2- 172.21.15.90:0/2616641942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77007d320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 -- 172.21.15.90:0/2616641942 >> 172.21.15.90:0/2616641942 conn(0x7fd770104350 msgr2=0x7fd770106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 -- 172.21.15.90:0/2616641942 shutdown_connections 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.014+0000 7fd776a00700 1 -- 172.21.15.90:0/2616641942 wait complete. 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.015+0000 7fd776a00700 1 Processor -- start 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd776a00700 1 -- start start 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd776a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd776a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010f380 con 0x7fd77007eec0 2024-06-02T15:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd76fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd76fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35306/0 (socket says 172.21.15.90:35306) 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.016+0000 7fd76fe00700 1 -- 172.21.15.90:0/306524962 learned_addr learned my addr 172.21.15.90:0/306524962 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd76fe00700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77019f760 con 0x7fd77007eec0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd76fe00700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7600119f0 tx=0x7fd760011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7600039c0 con 0x7fd77007eec0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd776a00700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77019f9f0 con 0x7fd77007eec0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760003b60 con 0x7fd77007eec0 2024-06-02T15:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7600197a0 con 0x7fd77007eec0 2024-06-02T15:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.017+0000 7fd776a00700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701a2690 con 0x7fd77007eec0 2024-06-02T15:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.018+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd760019a20 con 0x7fd77007eec0 2024-06-02T15:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.019+0000 7fd76cc00700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.019+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd754041040 con 0x7fd75403e470 2024-06-02T15:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.019+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd760053900 con 0x7fd77007eec0 2024-06-02T15:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.019+0000 7fd76f400700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.019+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.220+0000 7fd76f400700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.220+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.621+0000 7fd76f400700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:52.621+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:53.422+0000 7fd76f400700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:53.422+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:55.024+0000 7fd76f400700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1560565398 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:55.024+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:57.539+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fd76002dcd0 con 0x7fd77007eec0 2024-06-02T15:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:57.539+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 msgr2=0x7fd754040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:57.539+0000 7fd76cc00700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.540+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fd760019d10 con 0x7fd77007eec0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.541+0000 7fd76cc00700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 0x7fd7540442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.541+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd754041040 con 0x7fd754041ef0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.542+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 0x7fd7540442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.543+0000 7fd76f400700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 0x7fd7540442e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd75c003a10 tx=0x7fd75c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.544+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd754041040 con 0x7fd754041ef0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.554+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd770064ba0 con 0x7fd754041ef0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd76cc00700 1 -- 172.21.15.90:0/306524962 <== mgr.14152 v2:172.21.15.90:6800/436264481 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd770064ba0 con 0x7fd754041ef0 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 msgr2=0x7fd7540442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 0x7fd7540442e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd75c003a10 tx=0x7fd75c0092b0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 msgr2=0x7fd77019f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7600119f0 tx=0x7fd760011ad0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 shutdown_connections 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd754041ef0 0x7fd7540442e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd77007eec0 0x7fd77019f220 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 --2- 172.21.15.90:0/306524962 >> [v2:172.21.15.90:6800/1560565398,v1:172.21.15.90:6801/1560565398] conn(0x7fd75403e470 0x7fd754040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.555+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 >> 172.21.15.90:0/306524962 conn(0x7fd770104350 msgr2=0x7fd770104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.556+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 shutdown_connections 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:29:58.556+0000 7fd766000700 1 -- 172.21.15.90:0/306524962 wait complete. 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-06-02T15:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:58] ENGINE Bus STARTING 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:58] ENGINE Serving on https://172.21.15.90:7150 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:58] ENGINE Serving on http://172.21.15.90:8765 2024-06-02T15:29:59.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:29:59 smithi090 ceph-mon[31595]: [02/Jun/2024:15:29:58] ENGINE Bus STARTED 2024-06-02T15:30:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:00 smithi090 ceph-mon[31595]: mgrmap e12: x(active, since 2s) 2024-06-02T15:30:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.428+0000 7f30da600700 1 Processor -- start 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.428+0000 7f30da600700 1 -- start start 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.428+0000 7f30da600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.428+0000 7f30da600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30d407ce70 con 0x7f30d4109810 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.429+0000 7f30d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.429+0000 7f30d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35376/0 (socket says 172.21.15.90:35376) 2024-06-02T15:30:01.635 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.429+0000 7f30d3400700 1 -- 172.21.15.90:0/1229034375 learned_addr learned my addr 172.21.15.90:0/1229034375 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.429+0000 7f30d3400700 1 -- 172.21.15.90:0/1229034375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d410c140 con 0x7f30d4109810 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.429+0000 7f30d3400700 1 --2- 172.21.15.90:0/1229034375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30c40098a0 tx=0x7f30c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9214e09ee8fcac59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.430+0000 7f30d2000700 1 -- 172.21.15.90:0/1229034375 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4004160 con 0x7f30d4109810 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.430+0000 7f30d2000700 1 -- 172.21.15.90:0/1229034375 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c4004300 con 0x7f30d4109810 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.430+0000 7f30d2000700 1 -- 172.21.15.90:0/1229034375 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4011a20 con 0x7f30d4109810 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.430+0000 7f30da600700 1 -- 172.21.15.90:0/1229034375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 msgr2=0x7f30d410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.430+0000 7f30da600700 1 --2- 172.21.15.90:0/1229034375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30c40098a0 tx=0x7f30c4009bb0 comp rx=0 tx=0).stop 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.431+0000 7f30da600700 1 -- 172.21.15.90:0/1229034375 shutdown_connections 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.431+0000 7f30da600700 1 --2- 172.21.15.90:0/1229034375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d410bc00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.431+0000 7f30da600700 1 -- 172.21.15.90:0/1229034375 >> 172.21.15.90:0/1229034375 conn(0x7f30d4103280 msgr2=0x7f30d41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.431+0000 7f30da600700 1 -- 172.21.15.90:0/1229034375 shutdown_connections 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.431+0000 7f30da600700 1 -- 172.21.15.90:0/1229034375 wait complete. 2024-06-02T15:30:01.636 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30da600700 1 Processor -- start 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30da600700 1 -- start start 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30da600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30da600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d140 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35380/0 (socket says 172.21.15.90:35380) 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.432+0000 7f30d3400700 1 -- 172.21.15.90:0/2480431601 learned_addr learned my addr 172.21.15.90:0/2480431601 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30d3400700 1 -- 172.21.15.90:0/2480431601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419f680 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30d3400700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4003aa0 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c401b8f0 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419f970 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.433+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4023870 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.434+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a25b0 con 0x7f30d4109810 2024-06-02T15:30:01.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.435+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f30c4023b70 con 0x7f30d4109810 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.435+0000 7f30cbe00700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 0x7f30b8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.435+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f30c4054ec0 con 0x7f30d4109810 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.435+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d4064ba0 con 0x7f30d4109810 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.436+0000 7f30d2a00700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 0x7f30b8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.436+0000 7f30d2a00700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 0x7f30b8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.442+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30c401f030 con 0x7f30d4109810 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.680+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f30d402e670 con 0x7f30b803e4c0 2024-06-02T15:30:01.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.927+0000 7f30cbe00700 1 -- 172.21.15.90:0/2480431601 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f30d402e670 con 0x7f30b803e4c0 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 msgr2=0x7f30b8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 0x7f30b8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 msgr2=0x7f30d419f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c4011aa0 comp rx=0 tx=0).stop 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 shutdown_connections 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30b803e4c0 0x7f30b8040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 --2- 172.21.15.90:0/2480431601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30d4109810 0x7f30d419f140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 >> 172.21.15.90:0/2480431601 conn(0x7f30d4103280 msgr2=0x7f30d4107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 shutdown_connections 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:00.930+0000 7f30da600700 1 -- 172.21.15.90:0/2480431601 wait complete. 2024-06-02T15:30:01.639 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-06-02T15:30:02.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:01 smithi090 ceph-mon[31595]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:02.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:02.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:02.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:03.946 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:03 smithi090 ceph-mon[31595]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:03.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:04.392 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iNZU2LZEGGYwse2fVNme9ujYR5vkpRqup0k5nqV/IcO3MWDd9C5oO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717342202, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-02T15:30:04.392 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.305+0000 7fcea5600700 1 Processor -- start 2024-06-02T15:30:04.392 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.305+0000 7fcea5600700 1 -- start start 2024-06-02T15:30:04.392 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.305+0000 7fcea5600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:04.392 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.306+0000 7fcea5600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcea007ce90 con 0x7fcea01075e0 2024-06-02T15:30:04.393 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.306+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:04.393 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.306+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40974/0 (socket says 172.21.15.90:40974) 2024-06-02T15:30:04.393 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.306+0000 7fce9ea00700 1 -- 172.21.15.90:0/2090499041 learned_addr learned my addr 172.21.15.90:0/2090499041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:04.393 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.306+0000 7fce9ea00700 1 -- 172.21.15.90:0/2090499041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea0107f40 con 0x7fcea01075e0 2024-06-02T15:30:04.393 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.307+0000 7fce9ea00700 1 --2- 172.21.15.90:0/2090499041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce900098a0 tx=0x7fce90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426d055afd4912f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:04.394 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.307+0000 7fce9d600700 1 -- 172.21.15.90:0/2090499041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90004160 con 0x7fcea01075e0 2024-06-02T15:30:04.394 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.307+0000 7fce9d600700 1 -- 172.21.15.90:0/2090499041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce90004300 con 0x7fcea01075e0 2024-06-02T15:30:04.394 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.307+0000 7fce9d600700 1 -- 172.21.15.90:0/2090499041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90011a20 con 0x7fcea01075e0 2024-06-02T15:30:04.395 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 -- 172.21.15.90:0/2090499041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 msgr2=0x7fcea0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 --2- 172.21.15.90:0/2090499041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce900098a0 tx=0x7fce90009bb0 comp rx=0 tx=0).stop 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 -- 172.21.15.90:0/2090499041 shutdown_connections 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 --2- 172.21.15.90:0/2090499041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea0107a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 -- 172.21.15.90:0/2090499041 >> 172.21.15.90:0/2090499041 conn(0x7fcea0103280 msgr2=0x7fcea01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 -- 172.21.15.90:0/2090499041 shutdown_connections 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.308+0000 7fcea5600700 1 -- 172.21.15.90:0/2090499041 wait complete. 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.309+0000 7fcea5600700 1 Processor -- start 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.309+0000 7fcea5600700 1 -- start start 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.309+0000 7fcea5600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.309+0000 7fcea5600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0108fb0 con 0x7fcea01075e0 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:04.396 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce9ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40978/0 (socket says 172.21.15.90:40978) 2024-06-02T15:30:04.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce9ea00700 1 -- 172.21.15.90:0/2483343678 learned_addr learned my addr 172.21.15.90:0/2483343678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:04.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce9ea00700 1 -- 172.21.15.90:0/2483343678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea019f660 con 0x7fcea01075e0 2024-06-02T15:30:04.397 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce9ea00700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce90000c00 tx=0x7fce90011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:04.399 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.310+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90003aa0 con 0x7fcea01075e0 2024-06-02T15:30:04.399 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.311+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9001b8f0 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.311+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea019f8f0 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.311+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce90023870 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.311+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea01a25e0 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.312+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea0064ba0 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.312+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fce9001b440 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.313+0000 7fce97400700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 0x7fce840409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.313+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fce90053eb0 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.313+0000 7fce9e000700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 0x7fce840409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.314+0000 7fce9e000700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 0x7fce840409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006fd0 tx=0x7fce8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.318+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce9001f030 con 0x7fcea01075e0 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.573+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcea01971c0 con 0x7fce8403e520 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.822+0000 7fce97400700 1 -- 172.21.15.90:0/2483343678 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcea01971c0 con 0x7fce8403e520 2024-06-02T15:30:04.400 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 msgr2=0x7fce840409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 0x7fce840409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006fd0 tx=0x7fce8c006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 msgr2=0x7fcea019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce90000c00 tx=0x7fce90011aa0 comp rx=0 tx=0).stop 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 shutdown_connections 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fce8403e520 0x7fce840409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 --2- 172.21.15.90:0/2483343678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcea01075e0 0x7fcea019f120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.825+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 >> 172.21.15.90:0/2483343678 conn(0x7fcea0103280 msgr2=0x7fcea0195aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.826+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 shutdown_connections 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:02.826+0000 7fcea5600700 1 -- 172.21.15.90:0/2483343678 wait complete. 2024-06-02T15:30:04.401 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-06-02T15:30:05.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:04 smithi090 ceph-mon[31595]: mgrmap e13: x(active, since 6s) 2024-06-02T15:30:05.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.028+0000 7f5582000700 1 Processor -- start 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.028+0000 7f5582000700 1 -- start start 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.029+0000 7f5582000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.029+0000 7f5582000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f557c108670 con 0x7f557c107c80 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.029+0000 7f557b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.029+0000 7f557b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40980/0 (socket says 172.21.15.90:40980) 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.029+0000 7f557b400700 1 -- 172.21.15.90:0/1786953472 learned_addr learned my addr 172.21.15.90:0/1786953472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.030+0000 7f557b400700 1 -- 172.21.15.90:0/1786953472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c108e80 con 0x7f557c107c80 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.030+0000 7f557b400700 1 --2- 172.21.15.90:0/1786953472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098a0 tx=0x7f556c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16961b6357a2a5f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.030+0000 7f557a000700 1 -- 172.21.15.90:0/1786953472 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f556c004160 con 0x7f557c107c80 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f557a000700 1 -- 172.21.15.90:0/1786953472 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c004300 con 0x7f557c107c80 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f557a000700 1 -- 172.21.15.90:0/1786953472 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f556c011a20 con 0x7f557c107c80 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 -- 172.21.15.90:0/1786953472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 msgr2=0x7f557c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 --2- 172.21.15.90:0/1786953472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098a0 tx=0x7f556c009bb0 comp rx=0 tx=0).stop 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 -- 172.21.15.90:0/1786953472 shutdown_connections 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 --2- 172.21.15.90:0/1786953472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c1080a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 -- 172.21.15.90:0/1786953472 >> 172.21.15.90:0/1786953472 conn(0x7f557c103280 msgr2=0x7f557c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.031+0000 7f5582000700 1 -- 172.21.15.90:0/1786953472 shutdown_connections 2024-06-02T15:30:05.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.032+0000 7f5582000700 1 -- 172.21.15.90:0/1786953472 wait complete. 2024-06-02T15:30:05.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.032+0000 7f5582000700 1 Processor -- start 2024-06-02T15:30:05.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f5582000700 1 -- start start 2024-06-02T15:30:05.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f5582000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:05.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f5582000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10c230 con 0x7f557c107c80 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f557b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f557b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40990/0 (socket says 172.21.15.90:40990) 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.033+0000 7f557b400700 1 -- 172.21.15.90:0/1484506753 learned_addr learned my addr 172.21.15.90:0/1484506753 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f557b400700 1 -- 172.21.15.90:0/1484506753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c193330 con 0x7f557c107c80 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f557b400700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f556c009520 tx=0x7f556c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f556c0039e0 con 0x7f557c107c80 2024-06-02T15:30:05.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c01b8f0 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f556c019770 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c191520 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.034+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c191a60 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.036+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c064ba0 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.036+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f556c01b440 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.036+0000 7f5573e00700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 0x7f5560040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.036+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f556c054030 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.037+0000 7f557aa00700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 0x7f5560040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.037+0000 7f557aa00700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 0x7f5560040a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.042+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f556c01f030 con 0x7f557c107c80 2024-06-02T15:30:05.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.274+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f557c196850 con 0x7f557c107c80 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.275+0000 7f5573e00700 1 -- 172.21.15.90:0/1484506753 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f556c0105e0 con 0x7f557c107c80 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 msgr2=0x7f5560040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 0x7f5560040a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 msgr2=0x7f557c192df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f556c009520 tx=0x7f556c011aa0 comp rx=0 tx=0).stop 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 shutdown_connections 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f556003e5b0 0x7f5560040a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 --2- 172.21.15.90:0/1484506753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f557c107c80 0x7f557c192df0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 >> 172.21.15.90:0/1484506753 conn(0x7f557c103280 msgr2=0x7f557c104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.278+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 shutdown_connections 2024-06-02T15:30:05.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:05.279+0000 7f5582000700 1 -- 172.21.15.90:0/1484506753 wait complete. 2024-06-02T15:30:05.933 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T15:30:05.933 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-06-02T15:30:05.957 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T15:30:05.958 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-06-02T15:30:05.958 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-06-02T15:30:05.958 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: Password: e4pqfak0fc 2024-06-02T15:30:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:05.962 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config directory 2024-06-02T15:30:05.966 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-06-02T15:30:06.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:05 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1484506753' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-02T15:30:07.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.581+0000 7f4249000700 1 Processor -- start 2024-06-02T15:30:07.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.581+0000 7f4249000700 1 -- start start 2024-06-02T15:30:07.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.582+0000 7f4249000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:07.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.582+0000 7f4249000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4244109760 con 0x7f4244108d70 2024-06-02T15:30:07.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.582+0000 7f4241e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:07.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.582+0000 7f4241e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41002/0 (socket says 172.21.15.90:41002) 2024-06-02T15:30:07.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.582+0000 7f4241e00700 1 -- 172.21.15.90:0/2701567935 learned_addr learned my addr 172.21.15.90:0/2701567935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:07.502 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.583+0000 7f4241e00700 1 -- 172.21.15.90:0/2701567935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4244109f70 con 0x7f4244108d70 2024-06-02T15:30:07.502 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.583+0000 7f4241e00700 1 --2- 172.21.15.90:0/2701567935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42340098a0 tx=0x7f4234009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f0f18bad433f50 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:07.502 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.584+0000 7f4240a00700 1 -- 172.21.15.90:0/2701567935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4234004160 con 0x7f4244108d70 2024-06-02T15:30:07.502 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.584+0000 7f4240a00700 1 -- 172.21.15.90:0/2701567935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4234004300 con 0x7f4244108d70 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.584+0000 7f4240a00700 1 -- 172.21.15.90:0/2701567935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4234011a20 con 0x7f4244108d70 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.584+0000 7f4249000700 1 -- 172.21.15.90:0/2701567935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 msgr2=0x7f4244109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.584+0000 7f4249000700 1 --2- 172.21.15.90:0/2701567935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42340098a0 tx=0x7f4234009bb0 comp rx=0 tx=0).stop 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.585+0000 7f4249000700 1 -- 172.21.15.90:0/2701567935 shutdown_connections 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.585+0000 7f4249000700 1 --2- 172.21.15.90:0/2701567935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f4244109190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.585+0000 7f4249000700 1 -- 172.21.15.90:0/2701567935 >> 172.21.15.90:0/2701567935 conn(0x7f42441043b0 msgr2=0x7f42441067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.585+0000 7f4249000700 1 -- 172.21.15.90:0/2701567935 shutdown_connections 2024-06-02T15:30:07.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.585+0000 7f4249000700 1 -- 172.21.15.90:0/2701567935 wait complete. 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4249000700 1 Processor -- start 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4249000700 1 -- start start 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4249000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4249000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f424410d320 con 0x7f4244108d70 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4241e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4241e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41006/0 (socket says 172.21.15.90:41006) 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.586+0000 7f4241e00700 1 -- 172.21.15.90:0/4010984987 learned_addr learned my addr 172.21.15.90:0/4010984987 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f4241e00700 1 -- 172.21.15.90:0/4010984987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424407f090 con 0x7f4244108d70 2024-06-02T15:30:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f4241e00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4234009520 tx=0x7f4234011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42340039e0 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423401b8f0 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f4249000700 1 -- 172.21.15.90:0/4010984987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424407d280 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.587+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4234019770 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.588+0000 7f4249000700 1 -- 172.21.15.90:0/4010984987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424407d7c0 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.589+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f423401b440 con 0x7f4244108d70 2024-06-02T15:30:07.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.589+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4244064ba0 con 0x7f4244108d70 2024-06-02T15:30:07.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.589+0000 7f423aa00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 0x7f4228040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:07.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.589+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4234054020 con 0x7f4244108d70 2024-06-02T15:30:07.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.590+0000 7f4241400700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 0x7f4228040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:07.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.590+0000 7f4241400700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 0x7f4228040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:07.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.596+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f423401f030 con 0x7f4244108d70 2024-06-02T15:30:07.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.826+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f42441a48c0 con 0x7f4244108d70 2024-06-02T15:30:07.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.827+0000 7f423aa00700 1 -- 172.21.15.90:0/4010984987 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4234016df0 con 0x7f4244108d70 2024-06-02T15:30:07.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.831+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 msgr2=0x7f4228040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.831+0000 7f422fe00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 0x7f4228040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.831+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 msgr2=0x7f424407eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.831+0000 7f422fe00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4234009520 tx=0x7f4234011aa0 comp rx=0 tx=0).stop 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 shutdown_connections 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f422803e560 0x7f4228040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 --2- 172.21.15.90:0/4010984987 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4244108d70 0x7f424407eb50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 >> 172.21.15.90:0/4010984987 conn(0x7f42441043b0 msgr2=0x7f4244104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 shutdown_connections 2024-06-02T15:30:07.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:06.832+0000 7f422fe00700 1 -- 172.21.15.90:0/4010984987 wait complete. 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.121+0000 7fb7f5a00700 1 Processor -- start 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.121+0000 7fb7f5a00700 1 -- start start 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.121+0000 7fb7f5a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.121+0000 7fb7f5a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010b970 con 0x7fb7f010af80 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.122+0000 7fb7eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.122+0000 7fb7eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41008/0 (socket says 172.21.15.90:41008) 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.122+0000 7fb7eea00700 1 -- 172.21.15.90:0/2439351592 learned_addr learned my addr 172.21.15.90:0/2439351592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.123+0000 7fb7eea00700 1 -- 172.21.15.90:0/2439351592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f010c180 con 0x7fb7f010af80 2024-06-02T15:30:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.123+0000 7fb7eea00700 1 --2- 172.21.15.90:0/2439351592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00098a0 tx=0x7fb7e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbf789f481644aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:09.165 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.123+0000 7fb7ed600700 1 -- 172.21.15.90:0/2439351592 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e0004160 con 0x7fb7f010af80 2024-06-02T15:30:09.165 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.123+0000 7fb7ed600700 1 -- 172.21.15.90:0/2439351592 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e0004300 con 0x7fb7f010af80 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.123+0000 7fb7ed600700 1 -- 172.21.15.90:0/2439351592 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e0011a20 con 0x7fb7f010af80 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2439351592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 msgr2=0x7fb7f010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2439351592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00098a0 tx=0x7fb7e0009bb0 comp rx=0 tx=0).stop 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2439351592 shutdown_connections 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2439351592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f010b3a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2439351592 >> 172.21.15.90:0/2439351592 conn(0x7fb7f007e3b0 msgr2=0x7fb7f007e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.124+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2439351592 shutdown_connections 2024-06-02T15:30:09.166 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.125+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2439351592 wait complete. 2024-06-02T15:30:09.167 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7f5a00700 1 Processor -- start 2024-06-02T15:30:09.167 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7f5a00700 1 -- start start 2024-06-02T15:30:09.167 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7f5a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:09.167 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7f5a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010f530 con 0x7fb7f010af80 2024-06-02T15:30:09.167 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:09.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41010/0 (socket says 172.21.15.90:41010) 2024-06-02T15:30:09.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.126+0000 7fb7eea00700 1 -- 172.21.15.90:0/2906213826 learned_addr learned my addr 172.21.15.90:0/2906213826 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:09.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7eea00700 1 -- 172.21.15.90:0/2906213826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f019f6c0 con 0x7fb7f010af80 2024-06-02T15:30:09.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7eea00700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009520 tx=0x7fb7e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:09.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e00039e0 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e001b8f0 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7e0019770 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f019f950 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.127+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01a2640 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.129+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f0064ba0 con 0x7fb7f010af80 2024-06-02T15:30:09.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.129+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb7e001b440 con 0x7fb7f010af80 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.129+0000 7fb7e7400700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 0x7fb7d4040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.130+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7e00540c0 con 0x7fb7f010af80 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.130+0000 7fb7ee000700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 0x7fb7d4040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.130+0000 7fb7ee000700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 0x7fb7d4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006fd0 tx=0x7fb7dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.135+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7e001f030 con 0x7fb7f010af80 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.478+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb7f019fe30 con 0x7fb7f010af80 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.481+0000 7fb7e7400700 1 -- 172.21.15.90:0/2906213826 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb7f019fe30 con 0x7fb7f010af80 2024-06-02T15:30:09.170 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 msgr2=0x7fb7d4040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 0x7fb7d4040a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006fd0 tx=0x7fb7dc006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 msgr2=0x7fb7f019f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009520 tx=0x7fb7e0011aa0 comp rx=0 tx=0).stop 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 shutdown_connections 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7d403e5b0 0x7fb7d4040a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:09.171 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 --2- 172.21.15.90:0/2906213826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f010af80 0x7fb7f019f180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:09.172 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 >> 172.21.15.90:0/2906213826 conn(0x7fb7f007e3b0 msgr2=0x7fb7f0108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:09.172 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 shutdown_connections 2024-06-02T15:30:09.172 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-06-02T15:30:08.484+0000 7fb7f5a00700 1 -- 172.21.15.90:0/2906213826 wait complete. 2024-06-02T15:30:09.172 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-02T15:30:09.172 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-06-02T15:30:09.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.174 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-06-02T15:30:09.174 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.174 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-02T15:30:09.174 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:09.174 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-06-02T15:30:09.275 INFO:tasks.cephadm:Fetching config... 2024-06-02T15:30:09.275 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:30:09.276 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-02T15:30:09.320 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-02T15:30:09.321 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:30:09.321 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-02T15:30:09.407 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-02T15:30:09.407 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:30:09.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-02T15:30:09.530 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-02T15:30:09.530 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:30:09.530 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-02T15:30:09.632 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-02T15:30:09.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T15:30:09.732 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:30:09.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:09 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2906213826' entity='client.admin' 2024-06-02T15:30:09.762 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T15:30:09.819 INFO:teuthology.orchestra.run.smithi120.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:30:09.831 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T15:30:09.871 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtMRv+szwgDCdtphLOt/PWR2VcF9OlnIUVwFAtawsKoevN9to6sfBKqo11aZCJChsptm+MugE0QFbEnVNb5f2y8AyDZB+Y+opPuf2yUHnQM48rXkEXSq/T472X+J/gDkI/snG7L9zVY8smSNgodO80d1mbhYTbf/c7Z80Eplv6Vdrl305/JEG5LgYX+gFn3wo70OIfpItxXINbqvrxdBUrmoFSZlOTzRzg9mBtU/+uBoimCUOtRPAJ6yDPOKyOLwL/Qd5AJOqIubIndiwggcUFA3NeLTdahlljGqd1HVOoonta0dHTRsc9IsJ2IWRKtWcxHiDfbH92KbfLf9zJPfDSeJTSle+0TWOW8pp94DSlQyMB8jtXkWlIQTc0V2InLTUyRFQk9r1l2QwcStXOd+aJHzMEK6mEa8vWPR3OBue8rDzSyFFUxcW5jN8LlsUp5ySl6qQxSEf13kJmJ8J1GrusR101Tutyga8333SY9fEsUBbPTfR1lcgkobT8v7rN1yk= ceph-935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:30:09.883 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-02T15:30:10.297 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:12.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 -- 172.21.15.90:0/3571772256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 msgr2=0x7f2ce0105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:12.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3571772256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce0105ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009910 tx=0x7f2cd0009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:12.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 -- 172.21.15.90:0/3571772256 shutdown_connections 2024-06-02T15:30:12.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3571772256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce0105ce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:12.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 -- 172.21.15.90:0/3571772256 >> 172.21.15.90:0/3571772256 conn(0x7f2ce00fd2a0 msgr2=0x7f2ce00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 -- 172.21.15.90:0/3571772256 shutdown_connections 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.859+0000 7f2ce5800700 1 -- 172.21.15.90:0/3571772256 wait complete. 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.860+0000 7f2ce5800700 1 Processor -- start 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2ce5800700 1 -- start start 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2ce5800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2ce5800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0108d50 con 0x7f2ce01038f0 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2cdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2cdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60404/0 (socket says 172.21.15.90:60404) 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.861+0000 7f2cdfe00700 1 -- 172.21.15.90:0/3509228046 learned_addr learned my addr 172.21.15.90:0/3509228046 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.862+0000 7f2cdfe00700 1 -- 172.21.15.90:0/3509228046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce019ab50 con 0x7f2ce01038f0 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.862+0000 7f2cdfe00700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005190 tx=0x7f2cd0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:12.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.863+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0027070 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.863+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd0022470 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.863+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd000f5f0 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.863+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce019ade0 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.863+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce019dad0 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.864+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2cd002a070 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.864+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce01094d0 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.865+0000 7f2cdcc00700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 0x7f2cc4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.865+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2cd001e030 con 0x7f2ce01038f0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.865+0000 7f2cdf400700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 0x7f2cc4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.866+0000 7f2cdf400700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 0x7f2cc4040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:12.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:12.871+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cd0013070 con 0x7f2ce01038f0 2024-06-02T15:30:13.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.104+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2ce002e670 con 0x7f2ce01038f0 2024-06-02T15:30:13.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.108+0000 7f2cdcc00700 1 -- 172.21.15.90:0/3509228046 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2cd00310c0 con 0x7f2ce01038f0 2024-06-02T15:30:13.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 msgr2=0x7f2cc4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:13.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 0x7f2cc4040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 msgr2=0x7f2ce019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005190 tx=0x7f2cd0004f80 comp rx=0 tx=0).stop 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 shutdown_connections 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc403e570 0x7f2cc4040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 --2- 172.21.15.90:0/3509228046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ce01038f0 0x7f2ce019a610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.112+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 >> 172.21.15.90:0/3509228046 conn(0x7f2ce00fd2a0 msgr2=0x7f2ce00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.113+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 shutdown_connections 2024-06-02T15:30:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:13.113+0000 7f2ce5800700 1 -- 172.21.15.90:0/3509228046 wait complete. 2024-06-02T15:30:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3509228046' entity='client.admin' 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:13.895 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-02T15:30:13.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-02T15:30:14.304 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:16.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.981+0000 7fb2daa00700 1 -- 172.21.15.90:0/465053198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 msgr2=0x7fb2d4105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:16.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.981+0000 7fb2daa00700 1 --2- 172.21.15.90:0/465053198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d4105610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009910 tx=0x7fb2c4009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:16.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.981+0000 7fb2daa00700 1 -- 172.21.15.90:0/465053198 shutdown_connections 2024-06-02T15:30:16.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.981+0000 7fb2daa00700 1 --2- 172.21.15.90:0/465053198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d4105610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.981+0000 7fb2daa00700 1 -- 172.21.15.90:0/465053198 >> 172.21.15.90:0/465053198 conn(0x7fb2d4100770 msgr2=0x7fb2d4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.982+0000 7fb2daa00700 1 -- 172.21.15.90:0/465053198 shutdown_connections 2024-06-02T15:30:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.982+0000 7fb2daa00700 1 -- 172.21.15.90:0/465053198 wait complete. 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2daa00700 1 Processor -- start 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2daa00700 1 -- start start 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2daa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d410b130 con 0x7fb2d41051f0 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60420/0 (socket says 172.21.15.90:60420) 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.983+0000 7fb2d3e00700 1 -- 172.21.15.90:0/4187277078 learned_addr learned my addr 172.21.15.90:0/4187277078 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.984+0000 7fb2d3e00700 1 -- 172.21.15.90:0/4187277078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d419ad80 con 0x7fb2d41051f0 2024-06-02T15:30:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.984+0000 7fb2d3e00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005800 tx=0x7fb2c4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.984+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2c4027070 con 0x7fb2d41051f0 2024-06-02T15:30:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.984+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c400be10 con 0x7fb2d41051f0 2024-06-02T15:30:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.984+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2c400f460 con 0x7fb2d41051f0 2024-06-02T15:30:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.985+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d419b010 con 0x7fb2d41051f0 2024-06-02T15:30:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.985+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d419dcb0 con 0x7fb2d41051f0 2024-06-02T15:30:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.986+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb2c402a070 con 0x7fb2d41051f0 2024-06-02T15:30:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.986+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d410b8b0 con 0x7fb2d41051f0 2024-06-02T15:30:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.987+0000 7fb2d0c00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 0x7fb2b8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.987+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb2c401e030 con 0x7fb2d41051f0 2024-06-02T15:30:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.987+0000 7fb2d3400700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 0x7fb2b8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.988+0000 7fb2d3400700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 0x7fb2b8040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0006fd0 tx=0x7fb2c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:16.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:16.995+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2c4013070 con 0x7fb2d41051f0 2024-06-02T15:30:17.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.231+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb2d402e670 con 0x7fb2b803e570 2024-06-02T15:30:17.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.237+0000 7fb2d0c00700 1 -- 172.21.15.90:0/4187277078 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2d402e670 con 0x7fb2b803e570 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 msgr2=0x7fb2b8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 0x7fb2b8040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0006fd0 tx=0x7fb2c0006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 msgr2=0x7fb2d419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4005800 tx=0x7fb2c4009f60 comp rx=0 tx=0).stop 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 shutdown_connections 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b803e570 0x7fb2b8040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:17.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 --2- 172.21.15.90:0/4187277078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d41051f0 0x7fb2d419a840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:17.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 >> 172.21.15.90:0/4187277078 conn(0x7fb2d4100770 msgr2=0x7fb2d4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:17.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 shutdown_connections 2024-06-02T15:30:17.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:17.240+0000 7fb2daa00700 1 -- 172.21.15.90:0/4187277078 wait complete. 2024-06-02T15:30:18.040 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi120 2024-06-02T15:30:18.040 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:30:18.040 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.conf 2024-06-02T15:30:18.060 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:30:18.060 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:18.117 INFO:tasks.cephadm:Adding host smithi120 to orchestrator... 2024-06-02T15:30:18.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi120 2024-06-02T15:30:18.521 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:30:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:18 smithi090 ceph-mon[31595]: Updating smithi090:/etc/ceph/ceph.conf 2024-06-02T15:30:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:19 smithi090 ceph-mon[31595]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:19 smithi090 ceph-mon[31595]: Updating smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:30:20.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:20 smithi090 ceph-mon[31595]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:20.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:20 smithi090 ceph-mon[31595]: Updating smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T15:30:20.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:20.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:20.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:20.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 -- 172.21.15.90:0/2933896284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 msgr2=0x7f04b4075c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 --2- 172.21.15.90:0/2933896284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b4075c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009910 tx=0x7f04a8009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 -- 172.21.15.90:0/2933896284 shutdown_connections 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 --2- 172.21.15.90:0/2933896284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b4075c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 -- 172.21.15.90:0/2933896284 >> 172.21.15.90:0/2933896284 conn(0x7f04b40fea80 msgr2=0x7f04b4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 -- 172.21.15.90:0/2933896284 shutdown_connections 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.519+0000 7f04bb400700 1 -- 172.21.15.90:0/2933896284 wait complete. 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.521+0000 7f04bb400700 1 Processor -- start 2024-06-02T15:30:20.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.521+0000 7f04bb400700 1 -- start start 2024-06-02T15:30:20.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.521+0000 7f04bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:20.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.521+0000 7f04bb400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04b410f690 con 0x7f04b4075860 2024-06-02T15:30:20.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:20.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60442/0 (socket says 172.21.15.90:60442) 2024-06-02T15:30:20.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b8a00700 1 -- 172.21.15.90:0/3536845793 learned_addr learned my addr 172.21.15.90:0/3536845793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:20.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b8a00700 1 -- 172.21.15.90:0/3536845793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b41a0db0 con 0x7f04b4075860 2024-06-02T15:30:20.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b8a00700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04a8006010 tx=0x7f04a8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.522+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8026070 con 0x7f04b4075860 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.523+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b41a1040 con 0x7f04b4075860 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.523+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a800bc00 con 0x7f04b4075860 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.523+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8003910 con 0x7f04b4075860 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.523+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b419aa30 con 0x7f04b4075860 2024-06-02T15:30:20.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.524+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b4076fa0 con 0x7f04b4075860 2024-06-02T15:30:20.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.524+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f04a8029070 con 0x7f04b4075860 2024-06-02T15:30:20.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.525+0000 7f04b1600700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 0x7f049c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:20.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.525+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f04a801e070 con 0x7f04b4075860 2024-06-02T15:30:20.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.525+0000 7f04b3e00700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 0x7f049c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:20.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.526+0000 7f04b3e00700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 0x7f049c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:20.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.530+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04a8013070 con 0x7f04b4075860 2024-06-02T15:30:20.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:20.769+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}) -- 0x7f04b419bf60 con 0x7f049c03e570 2024-06-02T15:30:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:21 smithi090 ceph-mon[31595]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:22 smithi090 ceph-mon[31595]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi120", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:23 smithi090 ceph-mon[31595]: Deploying cephadm binary to smithi120 2024-06-02T15:30:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:23 smithi090 ceph-mon[31595]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:25.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.402+0000 7f04b1600700 1 -- 172.21.15.90:0/3536845793 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f04b419bf60 con 0x7f049c03e570 2024-06-02T15:30:25.402 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi120' with addr '172.21.15.120' 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 msgr2=0x7f049c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 0x7f049c040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 msgr2=0x7f04b41a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04a8006010 tx=0x7f04a8005780 comp rx=0 tx=0).stop 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 shutdown_connections 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f049c03e570 0x7f049c040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 --2- 172.21.15.90:0/3536845793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b4075860 0x7f04b41a0870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.406+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 >> 172.21.15.90:0/3536845793 conn(0x7f04b40fea80 msgr2=0x7f04b4107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:25.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.407+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 shutdown_connections 2024-06-02T15:30:25.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:25.407+0000 7f04bb400700 1 -- 172.21.15.90:0/3536845793 wait complete. 2024-06-02T15:30:25.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:25 smithi090 ceph-mon[31595]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:26.217 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-02T15:30:26.647 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:26.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:26 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:26.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:26 smithi090 ceph-mon[31595]: Added host smithi120 2024-06-02T15:30:26.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:26 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:26.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:26 smithi090 ceph-mon[31595]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:26.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:26 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:27.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 -- 172.21.15.90:0/3962856878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 msgr2=0x7f1538077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 --2- 172.21.15.90:0/3962856878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f1538077e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f152c009910 tx=0x7f152c009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 -- 172.21.15.90:0/3962856878 shutdown_connections 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 --2- 172.21.15.90:0/3962856878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f1538077e50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 -- 172.21.15.90:0/3962856878 >> 172.21.15.90:0/3962856878 conn(0x7f1538100c40 msgr2=0x7f1538103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 -- 172.21.15.90:0/3962856878 shutdown_connections 2024-06-02T15:30:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.684+0000 7f153fa00700 1 -- 172.21.15.90:0/3962856878 wait complete. 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.685+0000 7f153fa00700 1 Processor -- start 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.685+0000 7f153fa00700 1 -- start start 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.685+0000 7f153fa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.685+0000 7f153fa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1538113a90 con 0x7f1538077a70 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.686+0000 7f153d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.686+0000 7f153d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49290/0 (socket says 172.21.15.90:49290) 2024-06-02T15:30:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.686+0000 7f153d000700 1 -- 172.21.15.90:0/196649797 learned_addr learned my addr 172.21.15.90:0/196649797 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.686+0000 7f153d000700 1 -- 172.21.15.90:0/196649797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819f4f0 con 0x7f1538077a70 2024-06-02T15:30:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.686+0000 7f153d000700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f152c0056d0 tx=0x7f152c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.687+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c026070 con 0x7f1538077a70 2024-06-02T15:30:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.687+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152c00bc00 con 0x7f1538077a70 2024-06-02T15:30:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.687+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c003920 con 0x7f1538077a70 2024-06-02T15:30:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.687+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819f780 con 0x7f1538077a70 2024-06-02T15:30:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.687+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819fbe0 con 0x7f1538077a70 2024-06-02T15:30:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.688+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380791b0 con 0x7f1538077a70 2024-06-02T15:30:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.688+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f152c029070 con 0x7f1538077a70 2024-06-02T15:30:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.689+0000 7f1535600700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 0x7f1520040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.689+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f152c01e070 con 0x7f1538077a70 2024-06-02T15:30:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.689+0000 7f1537e00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 0x7f1520040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.690+0000 7f1537e00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 0x7f1520040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.695+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f152c013070 con 0x7f1538077a70 2024-06-02T15:30:28.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.937+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15381a0380 con 0x7f152003e570 2024-06-02T15:30:28.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.940+0000 7f1535600700 1 -- 172.21.15.90:0/196649797 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f15381a0380 con 0x7f152003e570 2024-06-02T15:30:28.940 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:28.940 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}] 2024-06-02T15:30:28.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 msgr2=0x7f1520040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 0x7f1520040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 msgr2=0x7f153819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f152c0056d0 tx=0x7f152c0057b0 comp rx=0 tx=0).stop 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 shutdown_connections 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f152003e570 0x7f1520040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 --2- 172.21.15.90:0/196649797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1538077a70 0x7f153819efb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.943+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 >> 172.21.15.90:0/196649797 conn(0x7f1538100c40 msgr2=0x7f15381024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.944+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 shutdown_connections 2024-06-02T15:30:28.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:28.944+0000 7f153fa00700 1 -- 172.21.15.90:0/196649797 wait complete. 2024-06-02T15:30:29.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:29 smithi090 ceph-mon[31595]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:29.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:29 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:29.703 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-06-02T15:30:29.703 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:30:29.703 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-06-02T15:30:29.721 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:30:29.722 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:29.779 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-06-02T15:30:29.779 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch host add smithi135 2024-06-02T15:30:30.209 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:30.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:30 smithi090 ceph-mon[31595]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:30:32.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 -- 172.21.15.90:0/2896195915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 msgr2=0x7f91b0101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 --2- 172.21.15.90:0/2896195915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b0101f20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009910 tx=0x7f91a0009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 -- 172.21.15.90:0/2896195915 shutdown_connections 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 --2- 172.21.15.90:0/2896195915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b0101f20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 -- 172.21.15.90:0/2896195915 >> 172.21.15.90:0/2896195915 conn(0x7f91b00fd7a0 msgr2=0x7f91b00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 -- 172.21.15.90:0/2896195915 shutdown_connections 2024-06-02T15:30:32.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.262+0000 7f91b4a00700 1 -- 172.21.15.90:0/2896195915 wait complete. 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.263+0000 7f91b4a00700 1 Processor -- start 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.263+0000 7f91b4a00700 1 -- start start 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91b4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91b4a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010f4d0 con 0x7f91b0101b40 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49698/0 (socket says 172.21.15.90:49698) 2024-06-02T15:30:32.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91ae800700 1 -- 172.21.15.90:0/3537874323 learned_addr learned my addr 172.21.15.90:0/3537874323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.264+0000 7f91ae800700 1 -- 172.21.15.90:0/3537874323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b01a0c10 con 0x7f91b0101b40 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91ae800700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91a00056d0 tx=0x7f91a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a0026070 con 0x7f91b0101b40 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a000bc00 con 0x7f91b0101b40 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b01a0ea0 con 0x7f91b0101b40 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91a0003920 con 0x7f91b0101b40 2024-06-02T15:30:32.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.265+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b019a890 con 0x7f91b0101b40 2024-06-02T15:30:32.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.266+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f91a0029070 con 0x7f91b0101b40 2024-06-02T15:30:32.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.266+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b006ba70 con 0x7f91b0101b40 2024-06-02T15:30:32.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.267+0000 7f91a7400700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 0x7f91940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:32.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.267+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f91a001e070 con 0x7f91b0101b40 2024-06-02T15:30:32.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.267+0000 7f91ade00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 0x7f91940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:32.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.268+0000 7f91ade00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 0x7f91940409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:32.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.273+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91a0013070 con 0x7f91b0101b40 2024-06-02T15:30:32.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:32 smithi090 ceph-mon[31595]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-02T15:30:32.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:32.522+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}) -- 0x7f91b019bdc0 con 0x7f919403e510 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:34.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:34 smithi090 ceph-mon[31595]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:35 smithi090 ceph-mon[31595]: Deploying cephadm binary to smithi135 2024-06-02T15:30:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:35 smithi090 ceph-mon[31595]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:36 smithi090 ceph-mon[31595]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:37.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.272+0000 7f91a7400700 1 -- 172.21.15.90:0/3537874323 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f91b019bdc0 con 0x7f919403e510 2024-06-02T15:30:37.271 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-06-02T15:30:37.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 msgr2=0x7f91940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:37.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 0x7f91940409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f919c006fd0 tx=0x7f919c006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:37.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 msgr2=0x7f91b01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:37.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91a00056d0 tx=0x7f91a00057b0 comp rx=0 tx=0).stop 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 shutdown_connections 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f919403e510 0x7f91940409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 --2- 172.21.15.90:0/3537874323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91b0101b40 0x7f91b01a06d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 >> 172.21.15.90:0/3537874323 conn(0x7f91b00fd7a0 msgr2=0x7f91b00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 shutdown_connections 2024-06-02T15:30:37.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:37.276+0000 7f91b4a00700 1 -- 172.21.15.90:0/3537874323 wait complete. 2024-06-02T15:30:38.081 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-02T15:30:38.509 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:38.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:38.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:38 smithi090 ceph-mon[31595]: Added host smithi135 2024-06-02T15:30:38.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:38.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:39 smithi090 ceph-mon[31595]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:40.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 -- 172.21.15.90:0/2158304633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 msgr2=0x7fb67c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:40.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 --2- 172.21.15.90:0/2158304633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c1024d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00b0a0 tx=0x7fb66c00b3b0 comp rx=0 tx=0).stop 2024-06-02T15:30:40.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 -- 172.21.15.90:0/2158304633 shutdown_connections 2024-06-02T15:30:40.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 --2- 172.21.15.90:0/2158304633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c1024d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:40.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 -- 172.21.15.90:0/2158304633 >> 172.21.15.90:0/2158304633 conn(0x7fb67c0779d0 msgr2=0x7fb67c077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:40.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 -- 172.21.15.90:0/2158304633 shutdown_connections 2024-06-02T15:30:40.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.447+0000 7fb682400700 1 -- 172.21.15.90:0/2158304633 wait complete. 2024-06-02T15:30:40.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.448+0000 7fb682400700 1 Processor -- start 2024-06-02T15:30:40.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.448+0000 7fb682400700 1 -- start start 2024-06-02T15:30:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.449+0000 7fb682400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.449+0000 7fb682400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c10c350 con 0x7fb67c1020f0 2024-06-02T15:30:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.449+0000 7fb67b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.449+0000 7fb67b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49710/0 (socket says 172.21.15.90:49710) 2024-06-02T15:30:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.449+0000 7fb67b400700 1 -- 172.21.15.90:0/2890903789 learned_addr learned my addr 172.21.15.90:0/2890903789 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb67b400700 1 -- 172.21.15.90:0/2890903789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c199cc0 con 0x7fb67c1020f0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb67b400700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006130 tx=0x7fb66c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c021020 con 0x7fb67c1020f0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c0079b0 con 0x7fb67c1020f0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c199f50 con 0x7fb67c1020f0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb66c017040 con 0x7fb67c1020f0 2024-06-02T15:30:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.450+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c19a410 con 0x7fb67c1020f0 2024-06-02T15:30:40.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.451+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c1035b0 con 0x7fb67c1020f0 2024-06-02T15:30:40.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.452+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb66c024020 con 0x7fb67c1020f0 2024-06-02T15:30:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.452+0000 7fb678c00700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 0x7fb660040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.452+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb66c00f050 con 0x7fb67c1020f0 2024-06-02T15:30:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.453+0000 7fb672a00700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 0x7fb660040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.453+0000 7fb672a00700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 0x7fb660040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb664006fd0 tx=0x7fb664006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:40.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.458+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb66c02b020 con 0x7fb67c1020f0 2024-06-02T15:30:40.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.695+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb67c064ba0 con 0x7fb66003e570 2024-06-02T15:30:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.697+0000 7fb678c00700 1 -- 172.21.15.90:0/2890903789 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fb67c064ba0 con 0x7fb66003e570 2024-06-02T15:30:40.696 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:30:40.696 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.120", "hostname": "smithi120", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.700+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 msgr2=0x7fb660040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.700+0000 7fb682400700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 0x7fb660040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb664006fd0 tx=0x7fb664006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.700+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 msgr2=0x7fb67c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.700+0000 7fb682400700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006130 tx=0x7fb66c0097d0 comp rx=0 tx=0).stop 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 shutdown_connections 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb66003e570 0x7fb660040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 --2- 172.21.15.90:0/2890903789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67c1020f0 0x7fb67c199780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 >> 172.21.15.90:0/2890903789 conn(0x7fb67c0779d0 msgr2=0x7fb67c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 shutdown_connections 2024-06-02T15:30:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:40.701+0000 7fb682400700 1 -- 172.21.15.90:0/2890903789 wait complete. 2024-06-02T15:30:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:41 smithi090 ceph-mon[31595]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:41.450 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-02T15:30:41.450 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-02T15:30:41.870 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:30:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:42 smithi090 ceph-mon[31595]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:30:43.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:43 smithi090 ceph-mon[31595]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:43.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 -- 172.21.15.90:0/3217707164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 msgr2=0x7f68fc10fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:43.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 --2- 172.21.15.90:0/3217707164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc10fa30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009910 tx=0x7f68ec009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:43.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 -- 172.21.15.90:0/3217707164 shutdown_connections 2024-06-02T15:30:43.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 --2- 172.21.15.90:0/3217707164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc10fa30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:43.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 -- 172.21.15.90:0/3217707164 >> 172.21.15.90:0/3217707164 conn(0x7f68fc078dd0 msgr2=0x7f68fc07b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:43.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 -- 172.21.15.90:0/3217707164 shutdown_connections 2024-06-02T15:30:43.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.846+0000 7f6903200700 1 -- 172.21.15.90:0/3217707164 wait complete. 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.847+0000 7f6903200700 1 Processor -- start 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.847+0000 7f6903200700 1 -- start start 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.847+0000 7f6903200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.847+0000 7f6903200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc113ac0 con 0x7f68fc10f650 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.848+0000 7f68fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:43.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.848+0000 7f68fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54734/0 (socket says 172.21.15.90:54734) 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.848+0000 7f68fbe00700 1 -- 172.21.15.90:0/154249620 learned_addr learned my addr 172.21.15.90:0/154249620 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.848+0000 7f68fbe00700 1 -- 172.21.15.90:0/154249620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc19f460 con 0x7f68fc10f650 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.848+0000 7f68fbe00700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006010 tx=0x7f68ec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.849+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68ec026070 con 0x7f68fc10f650 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.849+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68ec00bc00 con 0x7f68fc10f650 2024-06-02T15:30:43.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.849+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc19f6f0 con 0x7f68fc10f650 2024-06-02T15:30:43.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.849+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68ec003910 con 0x7f68fc10f650 2024-06-02T15:30:43.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.849+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1a37f0 con 0x7f68fc10f650 2024-06-02T15:30:43.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.850+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f68ec029070 con 0x7f68fc10f650 2024-06-02T15:30:43.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.850+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc10acc0 con 0x7f68fc10f650 2024-06-02T15:30:43.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.851+0000 7f68f8c00700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 0x7f68e0040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:43.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.851+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f68ec01e070 con 0x7f68fc10f650 2024-06-02T15:30:43.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.851+0000 7f68fb400700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 0x7f68e0040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:43.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.852+0000 7f68fb400700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 0x7f68e0040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68e8006fd0 tx=0x7f68e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:43.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:43.857+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f68ec013070 con 0x7f68fc10f650 2024-06-02T15:30:44.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.094+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f68fc1a0280 con 0x7f68fc10f650 2024-06-02T15:30:44.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.264+0000 7f68f8c00700 1 -- 172.21.15.90:0/154249620 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f68ec00d340 con 0x7f68fc10f650 2024-06-02T15:30:44.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 msgr2=0x7f68e0040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 0x7f68e0040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f68e8006fd0 tx=0x7f68e8006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 msgr2=0x7f68fc19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006010 tx=0x7f68ec005780 comp rx=0 tx=0).stop 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 shutdown_connections 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68e003e570 0x7f68e0040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 --2- 172.21.15.90:0/154249620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68fc10f650 0x7f68fc19ef20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.269+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 >> 172.21.15.90:0/154249620 conn(0x7f68fc078dd0 msgr2=0x7f68fc0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:44.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.270+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 shutdown_connections 2024-06-02T15:30:44.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:30:44.270+0000 7f6903200700 1 -- 172.21.15.90:0/154249620 wait complete. 2024-06-02T15:30:44.269 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-02T15:30:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/154249620' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-02T15:30:44.981 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-06-02T15:30:44.981 INFO:tasks.cephadm:Adding mon.b on smithi120 2024-06-02T15:30:44.982 INFO:tasks.cephadm:Adding mon.c on smithi135 2024-06-02T15:30:44.982 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c' 2024-06-02T15:30:45.206 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/154249620' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:46.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- 172.21.15.135:0/3271972279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 msgr2=0x7fc1fc102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 --2- 172.21.15.135:0/3271972279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc102490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009910 tx=0x7fc1ec009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- 172.21.15.135:0/3271972279 shutdown_connections 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 --2- 172.21.15.135:0/3271972279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc102490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- 172.21.15.135:0/3271972279 >> 172.21.15.135:0/3271972279 conn(0x7fc1fc0fd960 msgr2=0x7fc1fc0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- 172.21.15.135:0/3271972279 shutdown_connections 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- 172.21.15.135:0/3271972279 wait complete. 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 Processor -- start 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.596+0000 7fc203aea700 1 -- start start 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.597+0000 7fc203aea700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:46.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.597+0000 7fc203aea700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc10c330 con 0x7fc1fc1020b0 2024-06-02T15:30:46.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.597+0000 7fc201886700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:46.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.597+0000 7fc201886700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.135:42122/0 (socket says 172.21.15.135:42122) 2024-06-02T15:30:46.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.597+0000 7fc201886700 1 -- 172.21.15.135:0/2189892771 learned_addr learned my addr 172.21.15.135:0/2189892771 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:30:46.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.598+0000 7fc201886700 1 -- 172.21.15.135:0/2189892771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc19eb60 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.598+0000 7fc201886700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0056d0 tx=0x7fc1ec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.599+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1ec026070 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.599+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc19ee50 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.599+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1987a0 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.599+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ec00b6f0 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.599+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1ec030060 con 0x7fc1fc1020b0 2024-06-02T15:30:46.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.600+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0005330 con 0x7fc1fc1020b0 2024-06-02T15:30:46.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.600+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc1ec029070 con 0x7fc1fc1020b0 2024-06-02T15:30:46.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.601+0000 7fc1f2ffd700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 0x7fc1e80409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:46.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.601+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc1ec01e070 con 0x7fc1fc1020b0 2024-06-02T15:30:46.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.601+0000 7fc201085700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 0x7fc1e80409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:46.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.602+0000 7fc201085700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 0x7fc1e80409b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:46.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.618+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1ec013070 con 0x7fc1fc1020b0 2024-06-02T15:30:46.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.829+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}) -- 0x7fc1e0000c80 con 0x7fc1e803e4f0 2024-06-02T15:30:46.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.844+0000 7fc1f2ffd700 1 -- 172.21.15.135:0/2189892771 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc1e0000c80 con 0x7fc1e803e4f0 2024-06-02T15:30:46.843 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mon update... 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 msgr2=0x7fc1e80409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 0x7fc1e80409b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006fd0 tx=0x7fc1f8006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 msgr2=0x7fc1fc19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0056d0 tx=0x7fc1ec0057b0 comp rx=0 tx=0).stop 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 shutdown_connections 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc1e803e4f0 0x7fc1e80409b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 --2- 172.21.15.135:0/2189892771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1fc1020b0 0x7fc1fc19e620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 >> 172.21.15.135:0/2189892771 conn(0x7fc1fc0fd960 msgr2=0x7fc1fc06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 shutdown_connections 2024-06-02T15:30:46.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:46.847+0000 7fc203aea700 1 -- 172.21.15.135:0/2189892771 wait complete. 2024-06-02T15:30:47.464 DEBUG:teuthology.orchestra.run.smithi120:mon.b> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.b.service 2024-06-02T15:30:47.467 DEBUG:teuthology.orchestra.run.smithi135:mon.c> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.c.service 2024-06-02T15:30:47.469 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T15:30:47.470 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:30:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:47 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:48.526 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:30:48.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:48 smithi090 ceph-mon[31595]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:30:48.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:48 smithi090 ceph-mon[31595]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c;count:3 2024-06-02T15:30:48.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:48 smithi090 ceph-mon[31595]: Deploying daemon mon.c on smithi135 2024-06-02T15:30:49.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:49 smithi090 ceph-mon[31595]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:51.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 -- 172.21.15.135:0/19126290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 msgr2=0x7faf80107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:51.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 --2- 172.21.15.135:0/19126290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf80107210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf70009910 tx=0x7faf70009c20 comp rx=0 tx=0).stop 2024-06-02T15:30:51.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 -- 172.21.15.135:0/19126290 shutdown_connections 2024-06-02T15:30:51.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 --2- 172.21.15.135:0/19126290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf80107210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 -- 172.21.15.135:0/19126290 >> 172.21.15.135:0/19126290 conn(0x7faf800785c0 msgr2=0x7faf800789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 -- 172.21.15.135:0/19126290 shutdown_connections 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.322+0000 7faf84f7d700 1 -- 172.21.15.135:0/19126290 wait complete. 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.323+0000 7faf84f7d700 1 Processor -- start 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.323+0000 7faf84f7d700 1 -- start start 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.323+0000 7faf84f7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:51.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.323+0000 7faf84f7d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faf8010c460 con 0x7faf80106e30 2024-06-02T15:30:51.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.324+0000 7faf7e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:51.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.324+0000 7faf7e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.135:42140/0 (socket says 172.21.15.135:42140) 2024-06-02T15:30:51.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.324+0000 7faf7e59c700 1 -- 172.21.15.135:0/4261066508 learned_addr learned my addr 172.21.15.135:0/4261066508 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:30:51.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.324+0000 7faf7e59c700 1 -- 172.21.15.135:0/4261066508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf80199cf0 con 0x7faf80106e30 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf7e59c700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf700056d0 tx=0x7faf700057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf70026070 con 0x7faf80106e30 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf80199f80 con 0x7faf80106e30 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf8019a3e0 con 0x7faf80106e30 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf7000bc00 con 0x7faf80106e30 2024-06-02T15:30:51.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.325+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf700037f0 con 0x7faf80106e30 2024-06-02T15:30:51.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.326+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf801035f0 con 0x7faf80106e30 2024-06-02T15:30:51.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.327+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7faf70029070 con 0x7faf80106e30 2024-06-02T15:30:51.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.327+0000 7faf6f7fe700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 0x7faf680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:51.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.327+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faf7001e070 con 0x7faf80106e30 2024-06-02T15:30:51.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.327+0000 7faf7dd9b700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 0x7faf680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:51.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.328+0000 7faf7dd9b700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 0x7faf680409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:51.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.343+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf70013070 con 0x7faf80106e30 2024-06-02T15:30:51.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:51 smithi090 ceph-mon[31595]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:51.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.642+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faf80064ba0 con 0x7faf80106e30 2024-06-02T15:30:51.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.643+0000 7faf6f7fe700 1 -- 172.21.15.135:0/4261066508 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7faf70034090 con 0x7faf80106e30 2024-06-02T15:30:51.641 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:30:51.641 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":1,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","modified":"2024-06-02T15:28:33.985461Z","created":"2024-06-02T15:28:33.985461Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 msgr2=0x7faf680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 0x7faf680409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74006e40 comp rx=0 tx=0).stop 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 msgr2=0x7faf801997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf700056d0 tx=0x7faf700057b0 comp rx=0 tx=0).stop 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 shutdown_connections 2024-06-02T15:30:51.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faf6803e510 0x7faf680409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:51.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 --2- 172.21.15.135:0/4261066508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf80106e30 0x7faf801997b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:51.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 >> 172.21.15.135:0/4261066508 conn(0x7faf800785c0 msgr2=0x7faf80108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:51.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 shutdown_connections 2024-06-02T15:30:51.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:51.645+0000 7faf84f7d700 1 -- 172.21.15.135:0/4261066508 wait complete. 2024-06-02T15:30:51.644 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 1 2024-06-02T15:30:53.470 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T15:30:53.470 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T15:30:53.714 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: mon.a calling monitor election 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: mon.c calling monitor election 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: monmap epoch 2 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: last_changed 2024-06-02T15:30:51.790033+0000 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: min_mon_release 19 (squid) 2024-06-02T15:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: election_strategy: 1 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: fsmap 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: mgrmap e13: x(active, since 59s) 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: overall HEALTH_OK 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:30:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:57.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:30:56 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:30:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:57 smithi090 ceph-mon[31595]: Deploying daemon mon.b on smithi120 2024-06-02T15:30:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/1863915442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 msgr2=0x7f4cb0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/1863915442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cb0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8102f90 tx=0x7f4cb800feb0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/1863915442 shutdown_connections 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/1863915442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cb0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/1863915442 >> 172.21.15.135:0/1863915442 conn(0x7f4cc80feb50 msgr2=0x7f4cc8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/1863915442 shutdown_connections 2024-06-02T15:30:58.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.189+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/1863915442 wait complete. 2024-06-02T15:30:58.189 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:30:57 smithi135 ceph-mon[36839]: Deploying daemon mon.b on smithi120 2024-06-02T15:30:58.189 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:30:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 Processor -- start 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 -- start start 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cc819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810afb0 con 0x7f4cb00035c0 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4ccd3ce700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc810ae30 con 0x7f4cc819ec50 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.191+0000 7f4cc67fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42582/0 (socket says 172.21.15.135:42582) 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 -- 172.21.15.135:0/3308530446 learned_addr learned my addr 172.21.15.135:0/3308530446 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 -- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 msgr2=0x7f4cc81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-02T15:30:58.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 -- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 msgr2=0x7f4cc81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T15:30:58.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T15:30:58.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc6ffd700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cc819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:58.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc67fc700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:30:58.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc6ffd700 1 -- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 msgr2=0x7f4cc81988d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc6ffd700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.192+0000 7f4cc6ffd700 1 -- 172.21.15.135:0/3308530446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc819f290 con 0x7f4cb00035c0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4cc6ffd700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cc819e710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8010010 tx=0x7f4cb80116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4cb8013960 con 0x7f4cb00035c0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc8198e10 con 0x7f4cb00035c0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81993a0 con 0x7f4cb00035c0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb80044e0 con 0x7f4cb00035c0 2024-06-02T15:30:58.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.193+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4cb80119a0 con 0x7f4cb00035c0 2024-06-02T15:30:58.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.194+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc81047a0 con 0x7f4cb00035c0 2024-06-02T15:30:58.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.195+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4cb8023070 con 0x7f4cb00035c0 2024-06-02T15:30:58.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.195+0000 7f4caffff700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 0x7f4cb4040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:30:58.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.195+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4cb8020050 con 0x7f4cb00035c0 2024-06-02T15:30:58.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.195+0000 7f4cc67fc700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 0x7f4cb4040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:30:58.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.196+0000 7f4cc67fc700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 0x7f4cb4040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009180 tx=0x7f4cbc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:30:58.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.210+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cb8028020 con 0x7f4cb00035c0 2024-06-02T15:30:58.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.496+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4cc8199e00 con 0x7f4cb00035c0 2024-06-02T15:30:58.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.497+0000 7f4caffff700 1 -- 172.21.15.135:0/3308530446 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4cb800f8b0 con 0x7f4cb00035c0 2024-06-02T15:30:58.495 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:30:58.495 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","modified":"2024-06-02T15:30:51.790033Z","created":"2024-06-02T15:28:33.985461Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 msgr2=0x7f4cb4040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 0x7f4cb4040770 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009180 tx=0x7f4cbc008040 comp rx=0 tx=0).stop 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 msgr2=0x7f4cc819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cc819e710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8010010 tx=0x7f4cb80116a0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 shutdown_connections 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4cb403e2b0 0x7f4cb4040770 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cb00035c0 0x7f4cc819e710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 --2- 172.21.15.135:0/3308530446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cc819ec50 0x7f4cc81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:30:58.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 >> 172.21.15.135:0/3308530446 conn(0x7f4cc80feb50 msgr2=0x7f4cc80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:30:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 shutdown_connections 2024-06-02T15:30:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:30:58.500+0000 7f4ccd3ce700 1 -- 172.21.15.135:0/3308530446 wait complete. 2024-06-02T15:30:58.499 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 2 2024-06-02T15:30:59.062 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:30:58 smithi135 ceph-mon[36839]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:59.063 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:30:58 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/3308530446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:30:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:58 smithi090 ceph-mon[31595]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:30:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:30:58 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/3308530446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:31:00.091 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T15:31:00.092 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T15:31:00.336 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:31:00.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 podman[36432]: 2024-06-02 15:31:00.538303476 +0000 UTC m=+0.118057005 container create ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-02T15:31:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 podman[36432]: 2024-06-02 15:31:00.732671295 +0000 UTC m=+0.312424824 container init ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-02T15:31:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 podman[36432]: 2024-06-02 15:31:00.735579619 +0000 UTC m=+0.315333149 container start ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b, RELEASE=main-b7328fe, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-02T15:31:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: set uid:gid to 167:167 (ceph:ceph) 2024-06-02T15:31:01.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev), process ceph-mon, pid 2 2024-06-02T15:31:01.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pidfile_write: ignore empty --pid-file 2024-06-02T15:31:01.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: load: jerasure load: lrc 2024-06-02T15:31:01.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: RocksDB version: 7.9.2 2024-06-02T15:31:01.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Git sha 0 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Compile date 2024-05-22 16:22:08 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: DB SUMMARY 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: DB Session ID: 1X5KNINMCXJYTU3IO42K 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: CURRENT file: CURRENT 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: IDENTITY file: IDENTITY 2024-06-02T15:31:01.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-06-02T15:31:01.062 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-02T15:31:01.062 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-06-02T15:31:01.062 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.error_if_exists: 0 2024-06-02T15:31:01.062 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.create_if_missing: 0 2024-06-02T15:31:01.062 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.paranoid_checks: 1 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.env: 0x55c4c8007cc0 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.fs: PosixFileSystem 2024-06-02T15:31:01.063 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.info_log: 0x55c4c9ccbda0 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_file_opening_threads: 16 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.statistics: (nil) 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.use_fsync: 0 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_log_file_size: 0 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-02T15:31:01.064 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.keep_log_file_num: 1000 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.recycle_log_file_num: 0 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_fallocate: 1 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_mmap_reads: 0 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_mmap_writes: 0 2024-06-02T15:31:01.065 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.use_direct_reads: 0 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.create_missing_column_families: 0 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.db_log_dir: 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.wal_dir: 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-02T15:31:01.066 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-02T15:31:01.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-02T15:31:01.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-02T15:31:01.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-02T15:31:01.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-02T15:31:01.067 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.advise_random_on_open: 1 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.db_write_buffer_size: 0 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.write_buffer_manager: 0x55c4c9c9d0e0 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-02T15:31:01.068 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.rate_limiter: (nil) 2024-06-02T15:31:01.069 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-02T15:31:01.069 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.wal_recovery_mode: 2 2024-06-02T15:31:01.069 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enable_thread_tracking: 0 2024-06-02T15:31:01.069 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enable_pipelined_write: 0 2024-06-02T15:31:01.069 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.unordered_write: 0 2024-06-02T15:31:01.070 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-02T15:31:01.070 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-02T15:31:01.070 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-02T15:31:01.070 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-02T15:31:01.070 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.row_cache: None 2024-06-02T15:31:01.071 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.wal_filter: None 2024-06-02T15:31:01.074 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-02T15:31:01.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_ingest_behind: 0 2024-06-02T15:31:01.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.two_write_queues: 0 2024-06-02T15:31:01.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.manual_wal_flush: 0 2024-06-02T15:31:01.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.wal_compression: 0 2024-06-02T15:31:01.075 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.atomic_flush: 0 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.log_readahead_size: 0 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-02T15:31:01.076 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.best_efforts_recovery: 0 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.allow_data_in_errors: 0 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.db_host_id: __hostname__ 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-02T15:31:01.077 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_background_jobs: 2 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_background_compactions: -1 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_subcompactions: 1 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-02T15:31:01.078 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_total_wal_size: 0 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_open_files: -1 2024-06-02T15:31:01.079 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bytes_per_sync: 0 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_readahead_size: 0 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_background_flushes: -1 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Compression algorithms supported: 2024-06-02T15:31:01.080 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kZSTD supported: 0 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kXpressCompression supported: 0 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kLZ4HCCompression supported: 1 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kZlibCompression supported: 1 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kSnappyCompression supported: 1 2024-06-02T15:31:01.081 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kLZ4Compression supported: 1 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: kBZip2Compression supported: 0 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-02T15:31:01.082 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.merge_operator: 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_filter: None 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_filter_factory: None 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.sst_partitioner_factory: None 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-02T15:31:01.083 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c4c9ccbf60) 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks: 1 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout: pin_top_level_index_and_filter: 1 2024-06-02T15:31:01.084 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_type: 0 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_index_type: 0 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_shortening: 1 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: checksum: 4 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: no_block_cache: 0 2024-06-02T15:31:01.085 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache: 0x55c4c9ccf1f0 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_name: BinnedLRUCache 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_options: 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: capacity : 536870912 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_shard_bits : 4 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: strict_capacity_limit : 0 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: high_pri_pool_ratio: 0.000 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_cache_compressed: (nil) 2024-06-02T15:31:01.086 INFO:journalctl@ceph.mon.b.smithi120.stdout: persistent_cache: (nil) 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size: 4096 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_size_deviation: 10 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_restart_interval: 16 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: index_block_restart_interval: 1 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: metadata_block_size: 4096 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: partition_filters: 0 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: use_delta_encoding: 1 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: filter_policy: bloomfilter 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: whole_key_filtering: 1 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: verify_compression: 0 2024-06-02T15:31:01.087 INFO:journalctl@ceph.mon.b.smithi120.stdout: read_amp_bytes_per_bit: 0 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: format_version: 5 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_index_compression: 1 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: block_align: 0 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: max_auto_readahead_size: 262144 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: prepopulate_block_cache: 0 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: initial_auto_readahead_size: 8192 2024-06-02T15:31:01.088 INFO:journalctl@ceph.mon.b.smithi120.stdout: num_file_reads_for_auto_readahead: 2 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.write_buffer_size: 33554432 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_write_buffer_number: 2 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression: NoCompression 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression: Disabled 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.prefix_extractor: nullptr 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.num_levels: 7 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-02T15:31:01.089 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-02T15:31:01.090 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.level: 32767 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.strategy: 0 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.enabled: false 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-02T15:31:01.091 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.target_file_size_base: 67108864 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-02T15:31:01.092 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.arena_block_size: 1048576 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.disable_auto_compactions: 0 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-02T15:31:01.093 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-02T15:31:01.094 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.inplace_update_support: 0 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.bloom_locality: 0 2024-06-02T15:31:01.095 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.max_successive_merges: 0 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.paranoid_file_checks: 0 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.force_consistency_checks: 1 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.report_bg_io_stats: 0 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.ttl: 2592000 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-02T15:31:01.096 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enable_blob_files: false 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.min_blob_size: 0 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_file_size: 268435456 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.blob_file_starting_level: 0 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-02T15:31:01.097 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2883ef7d-415c-468c-bd78-39e04a214548 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342260765237, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342260765650, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717342260, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2883ef7d-415c-468c-bd78-39e04a214548", "db_session_id": "1X5KNINMCXJYTU3IO42K", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717342260765728, "job": 1, "event": "recovery_finished"} 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-02T15:31:01.098 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c4c9dec000 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: DB pointer 0x55c4c9dd8000 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** DB Stats ** 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T15:31:01.099 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-02T15:31:01.100 INFO:journalctl@ceph.mon.b.smithi120.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** Compaction Stats [default] ** 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.101 INFO:journalctl@ceph.mon.b.smithi120.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-02T15:31:01.102 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.102 INFO:journalctl@ceph.mon.b.smithi120.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T15:31:01.102 INFO:journalctl@ceph.mon.b.smithi120.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-02T15:31:01.102 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-02T15:31:01.102 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-02T15:31:01.103 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache BinnedLRUCache@0x55c4c9ccf1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: using public_addr v2:172.21.15.120:0/0 -> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: starting mon.b rank -1 at public addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] at bind addrs [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(???) e0 preinit fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).mds e1 new map 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-02T15:31:01.104 INFO:journalctl@ceph.mon.b.smithi120.stdout: e1 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout: legacy client fscid: -1 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout: 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout: No filesystems configured 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-02T15:31:01.105 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Deploying cephadm binary to smithi120 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.106 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Added host smithi120 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.107 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi120", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:01.108 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi120:/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:01.109 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Deploying cephadm binary to smithi135 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Added host smithi135 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:01.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:31:01.111 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:01.112 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/154249620' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/154249620' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.113 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:01.114 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c;count:3 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Deploying daemon mon.c on smithi135 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.a calling monitor election 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.115 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.c calling monitor election 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: monmap epoch 2 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: last_changed 2024-06-02T15:30:51.790033+0000 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: min_mon_release 19 (squid) 2024-06-02T15:31:01.116 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: election_strategy: 1 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: fsmap 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mgrmap e13: x(active, since 59s) 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: overall HEALTH_OK 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.117 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: Deploying daemon mon.b on smithi120 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/3308530446' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:31:01.118 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 ceph-mon[36451]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-02T15:31:01.119 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 bash[36432]: ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 2024-06-02T15:31:01.119 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:00 smithi120 systemd[1]: Started Ceph mon.b for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: mon.c calling monitor election 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: mon.a calling monitor election 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: monmap epoch 3 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: min_mon_release 19 (squid) 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: election_strategy: 1 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: fsmap 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: mgrmap e13: x(active, since 68s) 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: overall HEALTH_OK 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 -- 172.21.15.135:0/4240083861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 msgr2=0x7fb608005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 --2- 172.21.15.135:0/4240083861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb608005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb62006b790 tx=0x7fb61000fe60 comp rx=0 tx=0).stop 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 -- 172.21.15.135:0/4240083861 shutdown_connections 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 --2- 172.21.15.135:0/4240083861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb608005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 -- 172.21.15.135:0/4240083861 >> 172.21.15.135:0/4240083861 conn(0x7fb6200fda80 msgr2=0x7fb6200ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 -- 172.21.15.135:0/4240083861 shutdown_connections 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.288+0000 7fb6262dd700 1 -- 172.21.15.135:0/4240083861 wait complete. 2024-06-02T15:31:06.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.289+0000 7fb6262dd700 1 Processor -- start 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: mon.c calling monitor election 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: mon.a calling monitor election 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:06.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: monmap epoch 3 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: min_mon_release 19 (squid) 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: election_strategy: 1 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: fsmap 2024-06-02T15:31:06.289 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: mgrmap e13: x(active, since 68s) 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: overall HEALTH_OK 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.290 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:06.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.289+0000 7fb6262dd700 1 -- start start 2024-06-02T15:31:06.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb62019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:06.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:06.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010af50 con 0x7fb6080035c0 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010add0 con 0x7fb62019ee40 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb6262dd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb62010b0d0 con 0x7fb620199000 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:58642/0 (socket says 172.21.15.135:58642) 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 -- 172.21.15.135:0/1793536098 learned_addr learned my addr 172.21.15.135:0/1793536098 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61f7fe700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61ffff700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb62019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 msgr2=0x7fb6201994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 msgr2=0x7fb6201994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T15:31:06.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61f7fe700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 msgr2=0x7fb620198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61f7fe700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 msgr2=0x7fb620198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61f7fe700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb62487a700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.290+0000 7fb61f7fe700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 msgr2=0x7fb6201994b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 msgr2=0x7fb620198ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 -- 172.21.15.135:0/1793536098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb620101e40 con 0x7fb6080035c0 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61ffff700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb62019e900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb62006b790 tx=0x7fb610003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61002b020 con 0x7fb6080035c0 2024-06-02T15:31:06.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61001d440 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6201020d0 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610022040 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.291+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb620102610 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.292+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb61002e020 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.292+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620064ba0 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.293+0000 7fb61d7fa700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 0x7fb60c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.293+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb610054f80 con 0x7fb6080035c0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.293+0000 7fb61f7fe700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 0x7fb60c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:06.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.294+0000 7fb61f7fe700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 0x7fb60c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb614009850 tx=0x7fb614008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:06.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.299+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb610035030 con 0x7fb6080035c0 2024-06-02T15:31:06.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.595+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb6201039f0 con 0x7fb6080035c0 2024-06-02T15:31:06.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.597+0000 7fb61d7fa700 1 -- 172.21.15.135:0/1793536098 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fb61001ca10 con 0x7fb6080035c0 2024-06-02T15:31:06.595 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-06-02T15:31:06.595 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","modified":"2024-06-02T15:31:00.793433Z","created":"2024-06-02T15:28:33.985461Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:3300","nonce":0},{"type":"v1","addr":"172.21.15.120:6789","nonce":0}]},"addr":"172.21.15.120:6789/0","public_addr":"172.21.15.120:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-02T15:31:06.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 msgr2=0x7fb60c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:06.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 0x7fb60c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb614009850 tx=0x7fb614008040 comp rx=0 tx=0).stop 2024-06-02T15:31:06.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 msgr2=0x7fb62019e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb62019e900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb62006b790 tx=0x7fb610003cd0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 shutdown_connections 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb60c03e9a0 0x7fb60c040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb620199000 0x7fb6201994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6080035c0 0x7fb62019e900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 --2- 172.21.15.135:0/1793536098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb62019ee40 0x7fb620198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 >> 172.21.15.135:0/1793536098 conn(0x7fb6200fda80 msgr2=0x7fb6201080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 shutdown_connections 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:06.599+0000 7fb6262dd700 1 -- 172.21.15.135:0/1793536098 wait complete. 2024-06-02T15:31:06.598 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 3 2024-06-02T15:31:07.143 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:06 smithi135 ceph-mon[36839]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:07.145 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:07.145 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:06 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/1793536098' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:31:07.145 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:07.147 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-02T15:31:07.147 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-02T15:31:07.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:06 smithi090 ceph-mon[31595]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:07.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:07.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:06 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/1793536098' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:31:07.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:07.573 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:31:08.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: mon.c calling monitor election 2024-06-02T15:31:08.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:31:08.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: mon.a calling monitor election 2024-06-02T15:31:08.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: monmap epoch 3 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:08.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: min_mon_release 19 (squid) 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: election_strategy: 1 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: fsmap 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: mgrmap e13: x(active, since 68s) 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: overall HEALTH_OK 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:08.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:08.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/1793536098' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T15:31:08.061 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mon.b calling monitor election 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mon.b calling monitor election 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mon.c calling monitor election 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mon.a calling monitor election 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: monmap epoch 3 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: min_mon_release 19 (squid) 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: election_strategy: 1 2024-06-02T15:31:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: fsmap 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: mgrmap e13: x(active, since 70s) 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: overall HEALTH_OK 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: Updating smithi090:/etc/ceph/ceph.conf 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-02T15:31:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mon.b calling monitor election 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mon.b calling monitor election 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mon.c calling monitor election 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mon.a calling monitor election 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: monmap epoch 3 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:08.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: min_mon_release 19 (squid) 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: election_strategy: 1 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: fsmap 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: mgrmap e13: x(active, since 70s) 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: overall HEALTH_OK 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:08.981 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: Updating smithi090:/etc/ceph/ceph.conf 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:08.982 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mon.b calling monitor election 2024-06-02T15:31:09.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:09.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mon.b calling monitor election 2024-06-02T15:31:09.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mon.c calling monitor election 2024-06-02T15:31:09.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mon.a calling monitor election 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: monmap epoch 3 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: last_changed 2024-06-02T15:31:00.793433+0000 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: created 2024-06-02T15:28:33.985461+0000 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: min_mon_release 19 (squid) 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: election_strategy: 1 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: 2: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: fsmap 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T15:31:09.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: mgrmap e13: x(active, since 70s) 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: overall HEALTH_OK 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: Updating smithi090:/etc/ceph/ceph.conf 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: Updating smithi120:/etc/ceph/ceph.conf 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: Updating smithi135:/etc/ceph/ceph.conf 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: Updating smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: Updating smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.730+0000 7fc515c00700 1 -- 172.21.15.90:0/944506481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 msgr2=0x7fc5101081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.730+0000 7fc515c00700 1 --2- 172.21.15.90:0/944506481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101081a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc500009910 tx=0x7fc500009c20 comp rx=0 tx=0).stop 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.730+0000 7fc515c00700 1 -- 172.21.15.90:0/944506481 shutdown_connections 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.730+0000 7fc515c00700 1 --2- 172.21.15.90:0/944506481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101081a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.730+0000 7fc515c00700 1 -- 172.21.15.90:0/944506481 >> 172.21.15.90:0/944506481 conn(0x7fc5100fcf00 msgr2=0x7fc5100ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.731+0000 7fc515c00700 1 -- 172.21.15.90:0/944506481 shutdown_connections 2024-06-02T15:31:09.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.731+0000 7fc515c00700 1 -- 172.21.15.90:0/944506481 wait complete. 2024-06-02T15:31:09.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.732+0000 7fc515c00700 1 Processor -- start 2024-06-02T15:31:09.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.732+0000 7fc515c00700 1 -- start start 2024-06-02T15:31:09.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.732+0000 7fc515c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:09.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.732+0000 7fc515c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 0x7fc51019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:09.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 0x7fc51019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:09.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d3d0 con 0x7fc510107dc0 2024-06-02T15:31:09.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d250 con 0x7fc51019b050 2024-06-02T15:31:09.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010d550 con 0x7fc5101a0f80 2024-06-02T15:31:09.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46766/0 (socket says 172.21.15.90:46766) 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 -- 172.21.15.90:0/2819275638 learned_addr learned my addr 172.21.15.90:0/2819275638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50f400700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 0x7fc51019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515200700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 0x7fc51019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 -- 172.21.15.90:0/2819275638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 msgr2=0x7fc51019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 0x7fc51019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 -- 172.21.15.90:0/2819275638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 msgr2=0x7fc51019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 0x7fc51019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50fe00700 1 -- 172.21.15.90:0/2819275638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101a77f0 con 0x7fc510107dc0 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc50f400700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 0x7fc51019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.733+0000 7fc515200700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 0x7fc51019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:31:09.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc50fe00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc500005b40 tx=0x7fc500004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:09.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500026070 con 0x7fc510107dc0 2024-06-02T15:31:09.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc50000b750 con 0x7fc510107dc0 2024-06-02T15:31:09.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5101a7a20 con 0x7fc510107dc0 2024-06-02T15:31:09.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500009e60 con 0x7fc510107dc0 2024-06-02T15:31:09.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.734+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5101a7f50 con 0x7fc510107dc0 2024-06-02T15:31:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.736+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc510104500 con 0x7fc510107dc0 2024-06-02T15:31:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.736+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fc500029070 con 0x7fc510107dc0 2024-06-02T15:31:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.736+0000 7fc50cc00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 0x7fc4f4040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.736+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc500053350 con 0x7fc510107dc0 2024-06-02T15:31:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.737+0000 7fc50f400700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 0x7fc4f4040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.737+0000 7fc50f400700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 0x7fc4f4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009ec0 tx=0x7fc4fc009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:09.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.742+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc500013070 con 0x7fc510107dc0 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: Updating smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:09.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:09.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: Updating smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:09.906 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: Updating smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T15:31:09.906 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:31:09.906 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:09.907 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:09.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.972+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc5101a8150 con 0x7fc510107dc0 2024-06-02T15:31:09.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.973+0000 7fc50cc00700 1 -- 172.21.15.90:0/2819275638 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fc500030090 con 0x7fc510107dc0 2024-06-02T15:31:09.971 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:09.971 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-06-02T15:31:09.971 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:31:09.971 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 msgr2=0x7fc4f4040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 0x7fc4f4040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009ec0 tx=0x7fc4fc009210 comp rx=0 tx=0).stop 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 msgr2=0x7fc5101a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc500005b40 tx=0x7fc500004f80 comp rx=0 tx=0).stop 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 shutdown_connections 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc4f403e9a0 0x7fc4f4040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc51019b050 0x7fc51019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc510107dc0 0x7fc5101a0a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 --2- 172.21.15.90:0/2819275638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5101a0f80 0x7fc51019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 >> 172.21.15.90:0/2819275638 conn(0x7fc5100fcf00 msgr2=0x7fc51010a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 shutdown_connections 2024-06-02T15:31:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:09.976+0000 7fc515c00700 1 -- 172.21.15.90:0/2819275638 wait complete. 2024-06-02T15:31:10.730 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-02T15:31:10.730 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:31:10.730 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T15:31:10.812 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:31:10.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:31:10.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:10 smithi090 ceph-mon[31595]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T15:31:10.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:10 smithi090 ceph-mon[31595]: Reconfiguring daemon mon.a on smithi090 2024-06-02T15:31:10.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:10 smithi090 ceph-mon[31595]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:10.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:10 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2819275638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:10.934 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:31:10.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T15:31:10.960 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:10 smithi120 ceph-mon[36451]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T15:31:10.961 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:10 smithi120 ceph-mon[36451]: Reconfiguring daemon mon.a on smithi090 2024-06-02T15:31:10.961 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:10 smithi120 ceph-mon[36451]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:10.961 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:10 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2819275638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:10.966 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:31:10.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:31:11.033 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:31:11.033 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T15:31:11.060 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:10 smithi135 ceph-mon[36839]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T15:31:11.060 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:10 smithi135 ceph-mon[36839]: Reconfiguring daemon mon.a on smithi090 2024-06-02T15:31:11.060 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:10 smithi135 ceph-mon[36839]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:11.060 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:10 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2819275638' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:11.065 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:31:11.066 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T15:31:11.132 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-06-02T15:31:11.133 INFO:tasks.cephadm:Adding mgr.y on smithi120 2024-06-02T15:31:11.133 INFO:tasks.cephadm:Adding mgr.z on smithi135 2024-06-02T15:31:11.133 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi090=x;smithi120=y;smithi135=z' 2024-06-02T15:31:11.410 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:31:12.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- 172.21.15.135:0/3070567311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1030e0 msgr2=0x7f035c103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:12.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 --2- 172.21.15.135:0/3070567311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1030e0 0x7f035c103500 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0350009910 tx=0x7f0350009c20 comp rx=0 tx=0).stop 2024-06-02T15:31:12.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- 172.21.15.135:0/3070567311 shutdown_connections 2024-06-02T15:31:12.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 --2- 172.21.15.135:0/3070567311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1030e0 0x7f035c103500 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:12.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- 172.21.15.135:0/3070567311 >> 172.21.15.135:0/3070567311 conn(0x7f035c0fe680 msgr2=0x7f035c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:12.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- 172.21.15.135:0/3070567311 shutdown_connections 2024-06-02T15:31:12.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- 172.21.15.135:0/3070567311 wait complete. 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 Processor -- start 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- start start 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035c1030e0 0x7f035c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c0762f0 0x7f035c1a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f035c109020 con 0x7f035c0762f0 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f035c108ea0 con 0x7f035c1030e0 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.885+0000 7f036292d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f035c1091a0 con 0x7f035c075930 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51914/0 (socket says 172.21.15.135:51914) 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 -- 172.21.15.135:0/2325458133 learned_addr learned my addr 172.21.15.135:0/2325458133 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035bfff700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035c1030e0 0x7f035c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:12.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 -- 172.21.15.135:0/2325458133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035c1030e0 msgr2=0x7f035c0772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035c1030e0 0x7f035c0772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 -- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c0762f0 msgr2=0x7f035c1a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c0762f0 0x7f035c1a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 -- 172.21.15.135:0/2325458133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c1a2ec0 con 0x7f035c075930 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f035b7fe700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c00b780 tx=0x7f034c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c004230 con 0x7f035c075930 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c0043d0 con 0x7f035c075930 2024-06-02T15:31:12.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c1a31b0 con 0x7f035c075930 2024-06-02T15:31:12.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c011b20 con 0x7f035c075930 2024-06-02T15:31:12.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.886+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c1a36f0 con 0x7f035c075930 2024-06-02T15:31:12.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.887+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f034c0036a0 con 0x7f035c075930 2024-06-02T15:31:12.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.887+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c064ba0 con 0x7f035c075930 2024-06-02T15:31:12.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.888+0000 7f03597fa700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 0x7f0344040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:12.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.888+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f034c0522e0 con 0x7f035c075930 2024-06-02T15:31:12.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.888+0000 7f035bfff700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 0x7f0344040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:12.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.889+0000 7f035bfff700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 0x7f0344040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0350005190 tx=0x7f035001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:12.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:12.893+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f034c020740 con 0x7f035c075930 2024-06-02T15:31:13.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.110+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi120=y;smithi135=z", "target": ["mon-mgr", ""]}) -- 0x7f035c063980 con 0x7f034403e9a0 2024-06-02T15:31:13.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.124+0000 7f03597fa700 1 -- 172.21.15.135:0/2325458133 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f035c063980 con 0x7f034403e9a0 2024-06-02T15:31:13.123 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mgr update... 2024-06-02T15:31:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 msgr2=0x7f0344040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 0x7f0344040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0350005190 tx=0x7f035001a040 comp rx=0 tx=0).stop 2024-06-02T15:31:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 msgr2=0x7f035c075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c00b780 tx=0x7f034c00ba90 comp rx=0 tx=0).stop 2024-06-02T15:31:13.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 shutdown_connections 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f034403e9a0 0x7f0344040e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c0762f0 0x7f035c1a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f035c1030e0 0x7f035c0772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 --2- 172.21.15.135:0/2325458133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035c075930 0x7f035c075db0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 >> 172.21.15.135:0/2325458133 conn(0x7f035c0fe680 msgr2=0x7f035c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 shutdown_connections 2024-06-02T15:31:13.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:31:13.126+0000 7f036292d700 1 -- 172.21.15.135:0/2325458133 wait complete. 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: Reconfiguring mon.b (monmap changed)... 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:13.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:13 smithi120 ceph-mon[36451]: Reconfiguring daemon mon.b on smithi120 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: Reconfiguring mon.b (monmap changed)... 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:13 smithi135 ceph-mon[36839]: Reconfiguring daemon mon.b on smithi120 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: Reconfiguring mon.b (monmap changed)... 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:13 smithi090 ceph-mon[31595]: Reconfiguring daemon mon.b on smithi120 2024-06-02T15:31:13.676 DEBUG:teuthology.orchestra.run.smithi120:mgr.y> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-02T15:31:13.679 DEBUG:teuthology.orchestra.run.smithi135:mgr.z> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-02T15:31:13.681 INFO:tasks.cephadm:Deploying OSDs... 2024-06-02T15:31:13.681 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:31:13.682 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T15:31:13.738 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T15:31:13.739 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-06-02 15:30:14.120011825 +0000 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-06-02 15:24:14.310657035 +0000 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-06-02 15:24:14.310657035 +0000 2024-06-02T15:31:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-06-02 15:24:14.310657035 +0000 2024-06-02T15:31:13.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T15:31:13.871 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-06-02T15:31:13.871 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-06-02T15:31:13.871 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000384326 s, 1.3 MB/s 2024-06-02T15:31:13.875 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T15:31:13.931 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-06-02T15:31:13.980 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T15:31:13.980 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:13.980 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-06-02T15:31:13.980 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:13.980 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:13.981 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-06-02 15:30:14.170011066 +0000 2024-06-02T15:31:13.981 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-06-02 15:24:14.934647868 +0000 2024-06-02T15:31:13.981 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-06-02 15:24:14.934647868 +0000 2024-06-02T15:31:13.981 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-06-02 15:24:14.934647868 +0000 2024-06-02T15:31:13.981 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T15:31:14.053 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-06-02T15:31:14.054 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-06-02T15:31:14.054 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000401522 s, 1.3 MB/s 2024-06-02T15:31:14.057 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T15:31:14.110 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-06-02T15:31:14.161 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T15:31:14.161 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:14.161 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-06-02T15:31:14.161 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:14.162 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:14.162 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-06-02 15:30:14.214010397 +0000 2024-06-02T15:31:14.162 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-06-02 15:24:15.495639627 +0000 2024-06-02T15:31:14.162 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-06-02 15:24:15.495639627 +0000 2024-06-02T15:31:14.162 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-06-02 15:24:15.495639627 +0000 2024-06-02T15:31:14.162 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T15:31:14.240 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-06-02T15:31:14.240 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-06-02T15:31:14.240 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000342462 s, 1.5 MB/s 2024-06-02T15:31:14.243 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T15:31:14.296 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:14.344 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-06-02 15:30:14.262009668 +0000 2024-06-02T15:31:14.345 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-06-02 15:24:16.050631473 +0000 2024-06-02T15:31:14.345 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-06-02 15:24:16.050631473 +0000 2024-06-02T15:31:14.345 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-06-02 15:24:16.050631473 +0000 2024-06-02T15:31:14.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi120=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: Saving service mgr spec with placement smithi090=x;smithi120=y;smithi135=z;count:3 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:14.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-06-02T15:31:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-06-02T15:31:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000401369 s, 1.3 MB/s 2024-06-02T15:31:14.419 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi120=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: Saving service mgr spec with placement smithi090=x;smithi120=y;smithi135=z;count:3 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:14.436 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:14.467 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-06-02T15:31:14.467 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T15:31:14.485 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T15:31:14.485 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-06-02T15:31:14.541 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T15:31:14.541 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-02 15:31:00.235223597 +0000 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-02 15:22:28.492320826 +0000 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-02 15:22:28.492320826 +0000 2024-06-02T15:31:14.542 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-02 15:22:28.492320826 +0000 2024-06-02T15:31:14.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi120=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: Saving service mgr spec with placement smithi090=x;smithi120=y;smithi135=z;count:3 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T15:31:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T15:31:14.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:14.582 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-02T15:31:14.582 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-02T15:31:14.583 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000225335 s, 2.3 MB/s 2024-06-02T15:31:14.583 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T15:31:14.641 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-06-02T15:31:14.697 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T15:31:14.697 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:14.697 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-02T15:31:14.697 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:14.697 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:14.698 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-02 15:31:00.235223597 +0000 2024-06-02T15:31:14.698 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-02 15:22:28.839312976 +0000 2024-06-02T15:31:14.698 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-02 15:22:28.839312976 +0000 2024-06-02T15:31:14.698 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-02 15:22:28.839312976 +0000 2024-06-02T15:31:14.698 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T15:31:14.762 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-02T15:31:14.762 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-02T15:31:14.762 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000190766 s, 2.7 MB/s 2024-06-02T15:31:14.763 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T15:31:14.820 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-02 15:31:00.236223574 +0000 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-02 15:22:29.151305918 +0000 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-02 15:22:29.151305918 +0000 2024-06-02T15:31:14.876 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-02 15:22:29.151305918 +0000 2024-06-02T15:31:14.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T15:31:14.938 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-02T15:31:14.939 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-02T15:31:14.939 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000223111 s, 2.3 MB/s 2024-06-02T15:31:14.940 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T15:31:14.995 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-06-02T15:31:15.050 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-06-02 15:31:00.236223574 +0000 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-06-02 15:22:29.472298656 +0000 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-06-02 15:22:29.472298656 +0000 2024-06-02T15:31:15.051 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-06-02 15:22:29.472298656 +0000 2024-06-02T15:31:15.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T15:31:15.113 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-06-02T15:31:15.113 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-06-02T15:31:15.113 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00023837 s, 2.1 MB/s 2024-06-02T15:31:15.115 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T15:31:15.170 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-06-02T15:31:15.170 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T15:31:15.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T15:31:15.188 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-02 15:30:50.495389736 +0000 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-02 15:22:29.292976085 +0000 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-02 15:22:29.292976085 +0000 2024-06-02T15:31:15.243 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-02 15:22:29.292976085 +0000 2024-06-02T15:31:15.243 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T15:31:15.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:15 smithi135 ceph-mon[36839]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:15.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:15 smithi135 ceph-mon[36839]: Reconfiguring mon.c (monmap changed)... 2024-06-02T15:31:15.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:15 smithi135 ceph-mon[36839]: Reconfiguring daemon mon.c on smithi135 2024-06-02T15:31:15.305 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-02T15:31:15.305 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-02T15:31:15.306 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000248539 s, 2.1 MB/s 2024-06-02T15:31:15.307 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T15:31:15.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:15 smithi120 ceph-mon[36451]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:15.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:15 smithi120 ceph-mon[36451]: Reconfiguring mon.c (monmap changed)... 2024-06-02T15:31:15.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:15 smithi120 ceph-mon[36451]: Reconfiguring daemon mon.c on smithi135 2024-06-02T15:31:15.363 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-06-02T15:31:15.419 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T15:31:15.419 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-02 15:30:50.495389736 +0000 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-02 15:22:29.630968760 +0000 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-02 15:22:29.630968760 +0000 2024-06-02T15:31:15.420 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-02 15:22:29.630968760 +0000 2024-06-02T15:31:15.420 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T15:31:15.483 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-02T15:31:15.483 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-02T15:31:15.483 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000218709 s, 2.3 MB/s 2024-06-02T15:31:15.484 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T15:31:15.541 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-06-02T15:31:15.596 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-02 15:30:50.496389714 +0000 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-02 15:22:29.964961523 +0000 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-02 15:22:29.964961523 +0000 2024-06-02T15:31:15.597 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-02 15:22:29.964961523 +0000 2024-06-02T15:31:15.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T15:31:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:15 smithi090 ceph-mon[31595]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:15 smithi090 ceph-mon[31595]: Reconfiguring mon.c (monmap changed)... 2024-06-02T15:31:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:15 smithi090 ceph-mon[31595]: Reconfiguring daemon mon.c on smithi135 2024-06-02T15:31:15.659 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-02T15:31:15.660 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-02T15:31:15.660 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000188209 s, 2.7 MB/s 2024-06-02T15:31:15.661 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T15:31:15.717 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-06-02T15:31:15.772 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-06-02 15:30:50.496389714 +0000 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-06-02 15:22:30.290954459 +0000 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-06-02 15:22:30.290954459 +0000 2024-06-02T15:31:15.773 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-06-02 15:22:30.290954459 +0000 2024-06-02T15:31:15.774 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T15:31:15.834 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-06-02T15:31:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-06-02T15:31:15.835 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000174607 s, 2.9 MB/s 2024-06-02T15:31:15.836 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T15:31:15.892 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-06-02T15:31:15.893 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T15:31:16.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:16.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:16 smithi135 ceph-mon[36839]: Deploying daemon mgr.z on smithi135 2024-06-02T15:31:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:31:16.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:16 smithi090 ceph-mon[31595]: Deploying daemon mgr.z on smithi135 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:16 smithi120 ceph-mon[36451]: Deploying daemon mgr.z on smithi135 2024-06-02T15:31:17.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:17 smithi135 ceph-mon[36839]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:17.318 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 systemd[1]: Starting Ceph mgr.z for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:31:17.447 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:17 smithi090 ceph-mon[31595]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:17 smithi120 ceph-mon[36451]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:17.716 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 podman[38680]: 2024-06-02 15:31:17.366875845 +0000 UTC m=+0.016927143 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:31:17.716 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 podman[38680]: 2024-06-02 15:31:17.474942909 +0000 UTC m=+0.124994216 container create 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-06-02T15:31:17.716 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 podman[38680]: 2024-06-02 15:31:17.71892863 +0000 UTC m=+0.368979926 container init 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:31:18.058 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 podman[38680]: 2024-06-02 15:31:17.722406459 +0000 UTC m=+0.372457753 container start 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:31:18.058 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 ceph-mgr[38699]: -- 172.21.15.135:0/2827754716 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1460785a0 con 0x55c14604cc00 2024-06-02T15:31:18.059 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 bash[38680]: 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 2024-06-02T15:31:18.059 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 systemd[1]: Started Ceph mgr.z for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:31:18.059 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:17.874+0000 7f3dea371200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T15:31:18.059 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:18.060+0000 7f3dea371200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:31:18.317 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:18.163+0000 7f3dea371200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T15:31:18.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:18.471+0000 7f3dea371200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T15:31:18.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:18.555+0000 7f3dea371200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0333178 s, 315 MB/s 2024-06-02T15:31:18.978 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-06-02T15:31:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:18 smithi090 ceph-mon[31595]: Deploying daemon mgr.y on smithi120 2024-06-02T15:31:19.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:19.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.165 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:19.166 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:19.166 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:19.166 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:19.166 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:18 smithi120 ceph-mon[36451]: Deploying daemon mgr.y on smithi120 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T15:31:19.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:19.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:19.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:18 smithi135 ceph-mon[36839]: Deploying daemon mgr.y on smithi120 2024-06-02T15:31:19.812 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-06-02T15:31:20.058 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:19 smithi120 systemd[1]: Starting Ceph mgr.y for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:31:20.067 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:19.764+0000 7f3dea371200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T15:31:20.067 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:19.938+0000 7f3dea371200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T15:31:20.251 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 podman[37814]: 2024-06-02 15:31:19.998209621 +0000 UTC m=+0.017467927 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 podman[37814]: 2024-06-02 15:31:20.115586813 +0000 UTC m=+0.134845127 container create 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 podman[37814]: 2024-06-02 15:31:20.300955972 +0000 UTC m=+0.320214267 container init 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 podman[37814]: 2024-06-02 15:31:20.303592017 +0000 UTC m=+0.322850322 container start 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mgr[37833]: -- 172.21.15.120:0/1668402518 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600cac345a0 con 0x5600cac09000 2024-06-02T15:31:20.382 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 bash[37814]: 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 2024-06-02T15:31:20.405 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:20.115+0000 7f3dea371200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T15:31:20.638 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 systemd[1]: Started Ceph mgr.y for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:31:20.638 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:20.455+0000 7fda029b8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T15:31:20.638 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:20.640+0000 7fda029b8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:31:20.700 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:20.406+0000 7f3dea371200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T15:31:20.700 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:20.498+0000 7f3dea371200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T15:31:20.700 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:20.702+0000 7f3dea371200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:20.745+0000 7fda029b8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:20.902+0000 7f3dea371200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:21.558 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:21.075+0000 7fda029b8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T15:31:21.558 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:21.164+0000 7fda029b8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T15:31:21.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:21.536+0000 7f3dea371200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T15:31:21.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:21.768+0000 7f3dea371200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 -- 172.21.15.90:0/24225412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0106950 msgr2=0x7ff0d0108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/24225412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0106950 0x7ff0d0108d40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009990 tx=0x7ff0b8009ca0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 -- 172.21.15.90:0/24225412 shutdown_connections 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/24225412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0109280 0x7ff0d010b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/24225412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0104020 0x7ff0d0106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/24225412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0106950 0x7ff0d0108d40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.257+0000 7ff0d6c00700 1 -- 172.21.15.90:0/24225412 >> 172.21.15.90:0/24225412 conn(0x7ff0d00fda30 msgr2=0x7ff0d00ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/24225412 shutdown_connections 2024-06-02T15:31:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/24225412 wait complete. 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.259+0000 7ff0d6c00700 1 Processor -- start 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.259+0000 7ff0d6c00700 1 -- start start 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.259+0000 7ff0d6c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d6c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 0x7ff0d0101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d6c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0109280 0x7ff0d0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d6c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010f0a0 con 0x7ff0d0104020 2024-06-02T15:31:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d6c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010ef20 con 0x7ff0d0109280 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d6c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d010f220 con 0x7ff0d0106950 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58994/0 (socket says 172.21.15.90:58994) 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0c7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 0x7ff0d0101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 -- 172.21.15.90:0/1515984925 learned_addr learned my addr 172.21.15.90:0/1515984925 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0d4c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0109280 0x7ff0d0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 -- 172.21.15.90:0/1515984925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 msgr2=0x7ff0d0101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 0x7ff0d0101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 -- 172.21.15.90:0/1515984925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0109280 msgr2=0x7ff0d0102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0109280 0x7ff0d0102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0cfe00700 1 -- 172.21.15.90:0/1515984925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d0102dc0 con 0x7ff0d0104020 2024-06-02T15:31:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.260+0000 7ff0c7400700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 0x7ff0d0101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:31:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0cfe00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ef30 tx=0x7ff0c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00092e0 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c0004bb0 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a3850 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0004040 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.261+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a3d10 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.262+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff0c001b030 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.262+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d006a230 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.263+0000 7ff0cd600700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 0x7ff0b4040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.263+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff0c00525c0 con 0x7ff0d0104020 2024-06-02T15:31:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.263+0000 7ff0c7400700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 0x7ff0b4040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.263+0000 7ff0c7400700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 0x7ff0b4040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800b2e0 tx=0x7ff0b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:22.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.269+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0c0022020 con 0x7ff0d0104020 2024-06-02T15:31:22.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:22.516+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff0d0063980 con 0x7ff0b403e750 2024-06-02T15:31:22.567 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:22.199+0000 7f3dea371200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T15:31:22.567 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:22.295+0000 7f3dea371200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T15:31:22.767 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:22.430+0000 7fda029b8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T15:31:22.767 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:22.599+0000 7fda029b8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T15:31:22.767 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:22.768+0000 7fda029b8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T15:31:22.984 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:22.671+0000 7f3dea371200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:23.058+0000 7fda029b8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:23.149+0000 7fda029b8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T15:31:23.317 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.024+0000 7f3dea371200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T15:31:23.317 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.116+0000 7f3dea371200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T15:31:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T15:31:23.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T15:31:23.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:23.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:23.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:23.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:23.808 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:23.350+0000 7fda029b8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T15:31:23.808 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:23.541+0000 7fda029b8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T15:31:23.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.358+0000 7f3dea371200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T15:31:23.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.490+0000 7f3dea371200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T15:31:23.817 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.585+0000 7f3dea371200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T15:31:24.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-mon[36451]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T15:31:24.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-mon[36451]: Reconfiguring daemon mgr.x on smithi090 2024-06-02T15:31:24.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:23 smithi120 ceph-mon[36451]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:24.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:24.142+0000 7fda029b8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T15:31:24.317 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:23.933+0000 7f3dea371200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T15:31:24.317 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:24.025+0000 7f3dea371200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T15:31:24.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-mon[36839]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T15:31:24.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-mon[36839]: Reconfiguring daemon mgr.x on smithi090 2024-06-02T15:31:24.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:23 smithi135 ceph-mon[36839]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:24.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:23 smithi090 ceph-mon[31595]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T15:31:24.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:23 smithi090 ceph-mon[31595]: Reconfiguring daemon mgr.x on smithi090 2024-06-02T15:31:24.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:23 smithi090 ceph-mon[31595]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:24.785 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:24.362+0000 7fda029b8200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-06-02T15:31:24.995 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:24.623+0000 7f3dea371200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T15:31:24.995 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z[38695]: 2024-06-02T15:31:24.838+0000 7f3dea371200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:24.786+0000 7fda029b8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:24.882+0000 7fda029b8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: Standby manager daemon z started 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T15:31:25.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:24 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: Standby manager daemon z started 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T15:31:25.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:24 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: Standby manager daemon z started 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T15:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:24 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.135:0/2157982089' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:25.558 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:25.246+0000 7fda029b8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T15:31:25.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:25.561+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7ff0c0012050 con 0x7ff0d0104020 2024-06-02T15:31:25.928 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:25.597+0000 7fda029b8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T15:31:25.928 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:25.688+0000 7fda029b8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T15:31:26.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:25.929+0000 7fda029b8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T15:31:26.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:26.062+0000 7fda029b8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T15:31:26.308 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:26.156+0000 7fda029b8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T15:31:26.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-mon[36451]: mgrmap e14: x(active, since 88s), standbys: z 2024-06-02T15:31:26.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-mon[36451]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:26.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T15:31:26.808 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:26.497+0000 7fda029b8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T15:31:26.808 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:26.589+0000 7fda029b8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T15:31:26.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:26 smithi135 ceph-mon[36839]: mgrmap e14: x(active, since 88s), standbys: z 2024-06-02T15:31:26.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:26 smithi135 ceph-mon[36839]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:26.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:26 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T15:31:26.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:26 smithi090 ceph-mon[31595]: mgrmap e14: x(active, since 88s), standbys: z 2024-06-02T15:31:26.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:26 smithi090 ceph-mon[31595]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:26.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:26 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T15:31:27.558 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:27.190+0000 7fda029b8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T15:31:27.558 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y[37829]: 2024-06-02T15:31:27.406+0000 7fda029b8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T15:31:27.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2491882079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:27.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:27.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]': finished 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: Standby manager daemon y started 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T15:31:27.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:27 smithi135 ceph-mon[36839]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2491882079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]': finished 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: Standby manager daemon y started 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T15:31:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:27 smithi090 ceph-mon[31595]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2491882079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]: dispatch 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ad1d96f-8599-4ee1-9fd1-789f0be13368"}]': finished 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: Standby manager daemon y started 2024-06-02T15:31:28.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T15:31:28.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T15:31:28.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T15:31:28.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:27 smithi120 ceph-mon[36451]: from='mgr.? 172.21.15.120:0/2929053506' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T15:31:28.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:28.397+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff0c0019d30 con 0x7ff0d0104020 2024-06-02T15:31:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:28 smithi090 ceph-mon[31595]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:28 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3469604829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:28 smithi090 ceph-mon[31595]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-02T15:31:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:28 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T15:31:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:28 smithi120 ceph-mon[36451]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:28 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3469604829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:28 smithi120 ceph-mon[36451]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-02T15:31:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:28 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T15:31:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:28 smithi135 ceph-mon[36839]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:28 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3469604829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:28 smithi135 ceph-mon[36839]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-06-02T15:31:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:28 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T15:31:30.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:30 smithi090 ceph-mon[31595]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:30 smithi120 ceph-mon[36451]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:31.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:30 smithi135 ceph-mon[36839]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:32.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:32 smithi090 ceph-mon[31595]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:32 smithi120 ceph-mon[36451]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:33.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:32 smithi135 ceph-mon[36839]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:34.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:34 smithi090 ceph-mon[31595]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:35.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:34 smithi120 ceph-mon[36451]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:35.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:34 smithi135 ceph-mon[36839]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:36.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:36 smithi090 ceph-mon[31595]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:36.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T15:31:36.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:37.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:36 smithi120 ceph-mon[36451]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:37.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T15:31:37.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:37.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:36 smithi135 ceph-mon[36839]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:37.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T15:31:37.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:37.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:37 smithi090 ceph-mon[31595]: Deploying daemon osd.0 on smithi090 2024-06-02T15:31:38.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:37 smithi120 ceph-mon[36451]: Deploying daemon osd.0 on smithi090 2024-06-02T15:31:38.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:37 smithi135 ceph-mon[36839]: Deploying daemon osd.0 on smithi090 2024-06-02T15:31:38.804 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:38 smithi090 ceph-mon[31595]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:38.968 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:38 smithi135 ceph-mon[36839]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:38 smithi120 ceph-mon[36451]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:40.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:40 smithi090 ceph-mon[31595]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:40 smithi120 ceph-mon[36451]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:40 smithi135 ceph-mon[36839]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:42 smithi090 ceph-mon[31595]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:43.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:42 smithi120 ceph-mon[36451]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:43.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:42 smithi135 ceph-mon[36839]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:44.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:44 smithi090 ceph-mon[31595]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:44.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:44.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:44.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:44 smithi120 ceph-mon[36451]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:44 smithi135 ceph-mon[36839]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:46 smithi090 ceph-mon[31595]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:47.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:46 smithi120 ceph-mon[36451]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:47.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:46 smithi135 ceph-mon[36839]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:48 smithi120 ceph-mon[36451]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:48 smithi135 ceph-mon[36839]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T15:31:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:48 smithi090 ceph-mon[31595]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T15:31:49.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.253+0000 7ff0cd600700 1 -- 172.21.15.90:0/1515984925 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0d0063980 con 0x7ff0b403e750 2024-06-02T15:31:49.253 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-06-02T15:31:49.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 msgr2=0x7ff0b4040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 0x7ff0b4040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800b2e0 tx=0x7ff0b8005fb0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 msgr2=0x7ff0d0103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000ef30 tx=0x7ff0c000c3a0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 shutdown_connections 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff0b403e750 0x7ff0b4040c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff0d0109280 0x7ff0d0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d0104020 0x7ff0d0103980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 --2- 172.21.15.90:0/1515984925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0d0106950 0x7ff0d0101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:49.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 >> 172.21.15.90:0/1515984925 conn(0x7ff0d00fda30 msgr2=0x7ff0d0104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:49.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 shutdown_connections 2024-06-02T15:31:49.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:31:49.258+0000 7ff0d6c00700 1 -- 172.21.15.90:0/1515984925 wait complete. 2024-06-02T15:31:50.032 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.0.service 2024-06-02T15:31:50.037 INFO:tasks.cephadm:Deploying osd.1 on smithi120 with /dev/vg_nvme/lv_4... 2024-06-02T15:31:50.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:49 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:49 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:50.134 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T15:31:50.134 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T15:31:50.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:31:50.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:50.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:50.135 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:49 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:31:50.278 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:50 smithi120 ceph-mon[36451]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' 2024-06-02T15:31:51.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:31:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T15:31:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-mon[31595]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' 2024-06-02T15:31:51.102 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0[45806]: 2024-06-02T15:31:50.712+0000 7f938a800700 -1 osd.0 0 waiting for initial osdmap 2024-06-02T15:31:51.102 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:31:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0[45806]: 2024-06-02T15:31:50.733+0000 7f9380600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:51.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:50 smithi135 ceph-mon[36839]: from='osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562]' entity='osd.0' 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] boot 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T15:31:52.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:51 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] boot 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T15:31:52.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:51 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.166 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T15:31:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T15:31:52.167 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-02T15:31:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-02T15:31:52.167 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112622 s, 931 MB/s 2024-06-02T15:31:52.167 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: osd.0 [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] boot 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T15:31:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:51 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T15:31:52.739 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_4 2024-06-02T15:31:52.973 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:31:53.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:52 smithi120 ceph-mon[36451]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:53.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:52 smithi090 ceph-mon[31595]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:53.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:52 smithi135 ceph-mon[36839]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T15:31:54.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:53 smithi120 ceph-mon[36451]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T15:31:54.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:53 smithi090 ceph-mon[31595]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T15:31:54.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:53 smithi135 ceph-mon[36839]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T15:31:54.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 -- 172.21.15.120:0/3973174970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc104040 msgr2=0x7fb7dc10a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:54.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/3973174970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc104040 0x7fb7dc10a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b0d0 tx=0x7fb7d400b3e0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 -- 172.21.15.120:0/3973174970 shutdown_connections 2024-06-02T15:31:54.392 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/3973174970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc104040 0x7fb7dc10a8f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/3973174970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 0x7fb7dc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/3973174970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 -- 172.21.15.120:0/3973174970 >> 172.21.15.120:0/3973174970 conn(0x7fb7dc0fda40 msgr2=0x7fb7dc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 -- 172.21.15.120:0/3973174970 shutdown_connections 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.391+0000 7fb7e26e6700 1 -- 172.21.15.120:0/3973174970 wait complete. 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 Processor -- start 2024-06-02T15:31:54.393 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 -- start start 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 0x7fb7dc06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc104040 0x7fb7dc1a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10df90 con 0x7fb7dc104040 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10de10 con 0x7fb7dc1036a0 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7e26e6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7dc10e110 con 0x7fb7dc1024a0 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:56424/0 (socket says 172.21.15.120:56424) 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 -- 172.21.15.120:0/2987169888 learned_addr learned my addr 172.21.15.120:0/2987169888 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-02T15:31:54.394 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7dbfff700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 0x7fb7dc06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 -- 172.21.15.120:0/2987169888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 msgr2=0x7fb7dc06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 0x7fb7dc06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 -- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc104040 msgr2=0x7fb7dc1a2db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc104040 0x7fb7dc1a2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.392+0000 7fb7db7fe700 1 -- 172.21.15.120:0/2987169888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7dc1a3590 con 0x7fb7dc1036a0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7db7fe700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b530 tx=0x7fb7cc00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc015020 con 0x7fb7dc1036a0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc011ce0 con 0x7fb7dc1036a0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc010e40 con 0x7fb7dc1036a0 2024-06-02T15:31:54.395 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7dc1a3880 con 0x7fb7dc1036a0 2024-06-02T15:31:54.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.393+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7dc1a3e90 con 0x7fb7dc1036a0 2024-06-02T15:31:54.396 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.394+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7dc06a230 con 0x7fb7dc1036a0 2024-06-02T15:31:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.395+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb7cc018020 con 0x7fb7dc1036a0 2024-06-02T15:31:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.396+0000 7fb7d97fa700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 0x7fb7c40b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:31:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.396+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb7cc0e0490 con 0x7fb7dc1036a0 2024-06-02T15:31:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.397+0000 7fb7dbfff700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 0x7fb7c40b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:31:54.397 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.398+0000 7fb7dbfff700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 0x7fb7c40b4d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc103500 tx=0x7fb7d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:31:54.398 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.400+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7cc01f020 con 0x7fb7dc1036a0 2024-06-02T15:31:54.606 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:31:54.608+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb7dc063980 con 0x7fb7c40b2870 2024-06-02T15:31:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:54 smithi090 ceph-mon[31595]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:54 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:54 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:54 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:54 smithi120 ceph-mon[36451]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:54 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:55.236 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:54 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:55.237 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:54 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:55.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:54 smithi135 ceph-mon[36839]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:55.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:54 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:31:55.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:54 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:31:55.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:54 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi090 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 17089M 2024-06-02T15:31:56.025 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:56.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:56.026 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:56.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi090 2024-06-02T15:31:56.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:31:56.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 17089M 2024-06-02T15:31:56.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:56.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:56.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi090 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 17089M 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:31:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:31:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:31:57.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:56 smithi120 ceph-mon[36451]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:57.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:56 smithi090 ceph-mon[31595]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:57.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:56 smithi135 ceph-mon[36839]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/1808333844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]': finished 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:31:58.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:57 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/186435244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/1808333844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]': finished 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:31:58.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:57 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/186435244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:58.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/1808333844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]: dispatch 2024-06-02T15:31:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26a9dcac-5eab-4fac-90fa-534adccbf8c7"}]': finished 2024-06-02T15:31:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T15:31:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:31:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:57 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/186435244' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:31:59.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:31:59 smithi090 ceph-mon[31595]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:31:59 smithi120 ceph-mon[36451]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:31:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:31:59 smithi135 ceph-mon[36839]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:01 smithi120 ceph-mon[36451]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:01.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:01 smithi135 ceph-mon[36839]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:01 smithi090 ceph-mon[31595]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:03.472 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:03 smithi120 ceph-mon[36451]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:03.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:03 smithi135 ceph-mon[36839]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:03 smithi090 ceph-mon[31595]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:04.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T15:32:04.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:04.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T15:32:04.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:04.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T15:32:04.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:05 smithi120 ceph-mon[36451]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:05 smithi120 ceph-mon[36451]: Deploying daemon osd.1 on smithi120 2024-06-02T15:32:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:05 smithi135 ceph-mon[36839]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:05 smithi135 ceph-mon[36839]: Deploying daemon osd.1 on smithi120 2024-06-02T15:32:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:05 smithi090 ceph-mon[31595]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:05 smithi090 ceph-mon[31595]: Deploying daemon osd.1 on smithi120 2024-06-02T15:32:07.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:07 smithi120 ceph-mon[36451]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:07 smithi135 ceph-mon[36839]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:07 smithi090 ceph-mon[31595]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:09 smithi090 ceph-mon[31595]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:09.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:09 smithi120 ceph-mon[36451]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:09.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:09 smithi135 ceph-mon[36839]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:10.562 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.562 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.562 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:10.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:10.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:11 smithi120 ceph-mon[36451]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:11 smithi135 ceph-mon[36839]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:11 smithi090 ceph-mon[31595]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:12.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.347+0000 7fb7d97fa700 1 -- 172.21.15.120:0/2987169888 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb7dc063980 con 0x7fb7c40b2870 2024-06-02T15:32:12.346 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 1 on host 'smithi120' 2024-06-02T15:32:12.348 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 msgr2=0x7fb7c40b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 0x7fb7c40b4d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc103500 tx=0x7fb7d000b410 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 msgr2=0x7fb7dc1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b530 tx=0x7fb7cc00b840 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 shutdown_connections 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7c40b2870 0x7fb7c40b4d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7dc104040 0x7fb7dc1a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7dc1024a0 0x7fb7dc06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 --2- 172.21.15.120:0/2987169888 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7dc1036a0 0x7fb7dc1a0860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 >> 172.21.15.120:0/2987169888 conn(0x7fb7dc0fda40 msgr2=0x7fb7dc106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.350+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 shutdown_connections 2024-06-02T15:32:12.349 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:32:12.351+0000 7fb7e26e6700 1 -- 172.21.15.120:0/2987169888 wait complete. 2024-06-02T15:32:13.187 DEBUG:teuthology.orchestra.run.smithi120:osd.1> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.1.service 2024-06-02T15:32:13.189 INFO:tasks.cephadm:Deploying osd.2 on smithi135 with /dev/vg_nvme/lv_4... 2024-06-02T15:32:13.190 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T15:32:13.436 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:32:13.557 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:13 smithi135 ceph-mon[36839]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:13 smithi135 ceph-mon[36839]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:13 smithi135 ceph-mon[36839]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:13 smithi120 ceph-mon[36451]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:13 smithi120 ceph-mon[36451]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:13.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:13 smithi120 ceph-mon[36451]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:13 smithi090 ceph-mon[31595]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:13 smithi090 ceph-mon[31595]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:13 smithi090 ceph-mon[31595]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.488 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.489 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:14 smithi135 ceph-mon[36839]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-mon[36451]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:14.559 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1[41021]: 2024-06-02T15:32:14.331+0000 7fbce3709700 -1 osd.1 0 waiting for initial osdmap 2024-06-02T15:32:14.559 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1[41021]: 2024-06-02T15:32:14.335+0000 7fbcdf4fc700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:14 smithi090 ceph-mon[31595]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:15.307 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T15:32:15.308 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T15:32:15.308 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-02T15:32:15.308 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-02T15:32:15.308 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114839 s, 913 MB/s 2024-06-02T15:32:15.308 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-02T15:32:15.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:15 smithi120 ceph-mon[36451]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:32:15.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:15 smithi120 ceph-mon[36451]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T15:32:15.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:15 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:15 smithi090 ceph-mon[31595]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:32:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:15 smithi090 ceph-mon[31595]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T15:32:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:15 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:15.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:15 smithi135 ceph-mon[36839]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:32:15.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:15 smithi135 ceph-mon[36839]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T15:32:15.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:15 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:15.889 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-06-02T15:32:16.124 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] boot 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.538 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:16 smithi135 ceph-mon[36839]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:32:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:32:16.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] boot 2024-06-02T15:32:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T15:32:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:16 smithi120 ceph-mon[36451]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: osd.1 [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] boot 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T15:32:16.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:16 smithi090 ceph-mon[31595]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-02T15:32:17.444 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T15:32:17.444 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi120 2024-06-02T15:32:17.444 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.446 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T15:32:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi120 2024-06-02T15:32:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 -- 172.21.15.135:0/2142307032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e0101e40 msgr2=0x7fe3e0102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/2142307032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e0101e40 0x7fe3e0102260 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009990 tx=0x7fe3d0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 -- 172.21.15.135:0/2142307032 shutdown_connections 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/2142307032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e0103160 0x7fe3e01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/2142307032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e0101e40 0x7fe3e0102260 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/2142307032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e01027a0 0x7fe3e0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 -- 172.21.15.135:0/2142307032 >> 172.21.15.135:0/2142307032 conn(0x7fe3e00fda60 msgr2=0x7fe3e00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 -- 172.21.15.135:0/2142307032 shutdown_connections 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 -- 172.21.15.135:0/2142307032 wait complete. 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.665+0000 7fe3e5c32700 1 Processor -- start 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 -- start start 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e0101e40 0x7fe3e019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0103160 0x7fe3e01a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e01112b0 con 0x7fe3e0101e40 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0111130 con 0x7fe3e0103160 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0111430 con 0x7fe3e01027a0 2024-06-02T15:32:17.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46162/0 (socket says 172.21.15.135:46162) 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 -- 172.21.15.135:0/4054941743 learned_addr learned my addr 172.21.15.135:0/4054941743 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3dffff700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0103160 0x7fe3e01a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 -- 172.21.15.135:0/4054941743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0103160 msgr2=0x7fe3e01a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0103160 0x7fe3e01a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 -- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e0101e40 msgr2=0x7fe3e019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e0101e40 0x7fe3e019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 -- 172.21.15.135:0/4054941743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e01a3590 con 0x7fe3e01027a0 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3deffd700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400d9b0 tx=0x7fe3d400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:32:17.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4017070 con 0x7fe3e01027a0 2024-06-02T15:32:17.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d4005020 con 0x7fe3e01027a0 2024-06-02T15:32:17.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01a3880 con 0x7fe3e01027a0 2024-06-02T15:32:17.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.666+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4009b50 con 0x7fe3e01027a0 2024-06-02T15:32:17.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.667+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01a3df0 con 0x7fe3e01027a0 2024-06-02T15:32:17.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.667+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e006a230 con 0x7fe3e01027a0 2024-06-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.669+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe3d40051d0 con 0x7fe3e01027a0 2024-06-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.669+0000 7fe3dcff9700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 0x7fe3c80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.670+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe3d40e0cf0 con 0x7fe3e01027a0 2024-06-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.670+0000 7fe3df7fe700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 0x7fe3c80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:17.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.671+0000 7fe3df7fe700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 0x7fe3c80b4cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098e0 tx=0x7fe3d0003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:32:17.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.673+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3d4020020 con 0x7fe3e01027a0 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi120 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 17088M 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:17.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:17.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:17.888+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe3e0063980 con 0x7fe3c80b27f0 2024-06-02T15:32:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:18 smithi090 ceph-mon[31595]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:18 smithi090 ceph-mon[31595]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:18.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:18 smithi135 ceph-mon[36839]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:18.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:18 smithi135 ceph-mon[36839]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:18.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:18.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:18.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:18.710 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:18 smithi120 ceph-mon[36451]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:18.710 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:18 smithi120 ceph-mon[36451]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:18.711 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:18.711 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:18.711 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/506195532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]': finished 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T15:32:20.901 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/506195532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:21.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]': finished 2024-06-02T15:32:21.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T15:32:21.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/506195532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]: dispatch 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a650ebd-c0d3-4787-bf26-59e8c5ed88f8"}]': finished 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T15:32:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:22.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:21 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/1685960632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:22.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:21 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/1685960632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:22.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:21 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/1685960632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:22 smithi120 ceph-mon[36451]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:23.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:22 smithi135 ceph-mon[36839]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:22 smithi090 ceph-mon[31595]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:24.742 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:24 smithi135 ceph-mon[36839]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:25.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:24 smithi120 ceph-mon[36451]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:25.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:24 smithi090 ceph-mon[31595]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:26.980 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:26 smithi135 ceph-mon[36839]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:27.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:26 smithi120 ceph-mon[36451]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:26 smithi090 ceph-mon[31595]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:28.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T15:32:28.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T15:32:28.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:28.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T15:32:28.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:28.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:28 smithi135 ceph-mon[36839]: Deploying daemon osd.2 on smithi135 2024-06-02T15:32:28.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:28 smithi135 ceph-mon[36839]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:28 smithi120 ceph-mon[36451]: Deploying daemon osd.2 on smithi135 2024-06-02T15:32:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:28 smithi120 ceph-mon[36451]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:28 smithi090 ceph-mon[31595]: Deploying daemon osd.2 on smithi135 2024-06-02T15:32:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:28 smithi090 ceph-mon[31595]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:30.973 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:30 smithi120 ceph-mon[36451]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:31.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:30 smithi135 ceph-mon[36839]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:31.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:30 smithi090 ceph-mon[31595]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:32 smithi120 ceph-mon[36451]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:33.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:32 smithi135 ceph-mon[36839]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:32 smithi090 ceph-mon[31595]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:34.282 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.282 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.282 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:34.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:35.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:34 smithi135 ceph-mon[36839]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:35.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:34 smithi120 ceph-mon[36451]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:34 smithi090 ceph-mon[31595]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:36.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.171+0000 7fe3dcff9700 1 -- 172.21.15.135:0/4054941743 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3e0063980 con 0x7fe3c80b27f0 2024-06-02T15:32:36.171 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 2 on host 'smithi135' 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 msgr2=0x7fe3c80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 0x7fe3c80b4cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098e0 tx=0x7fe3d0003860 comp rx=0 tx=0).stop 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 msgr2=0x7fe3e019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400d9b0 tx=0x7fe3d400dcc0 comp rx=0 tx=0).stop 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 shutdown_connections 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3c80b27f0 0x7fe3c80b4cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:36.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3e0103160 0x7fe3e01a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3e0101e40 0x7fe3e019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 --2- 172.21.15.135:0/4054941743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3e01027a0 0x7fe3e019be70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 >> 172.21.15.135:0/4054941743 conn(0x7fe3e00fda60 msgr2=0x7fe3e0100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:32:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 shutdown_connections 2024-06-02T15:32:36.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:32:36.174+0000 7fe3e5c32700 1 -- 172.21.15.135:0/4054941743 wait complete. 2024-06-02T15:32:36.982 DEBUG:teuthology.orchestra.run.smithi135:osd.2> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.2.service 2024-06-02T15:32:36.985 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_3... 2024-06-02T15:32:36.985 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:37.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:37 smithi135 ceph-mon[36839]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:37 smithi090 ceph-mon[31595]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:37.415 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:37 smithi120 ceph-mon[36451]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:38.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:38 smithi090 ceph-mon[31595]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T15:32:38.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:38.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T15:32:38.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:38.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:38 smithi135 ceph-mon[36839]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: from='osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:39.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:38 smithi120 ceph-mon[36451]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:32:39.613 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-mon[36839]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:32:39.613 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-mon[36839]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T15:32:39.614 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:39.614 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-mon[36839]: from='osd.2 ' entity='osd.2' 2024-06-02T15:32:39.614 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:39.614 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2[41940]: 2024-06-02T15:32:39.343+0000 7f0dc73e6700 -1 osd.2 0 waiting for initial osdmap 2024-06-02T15:32:39.614 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2[41940]: 2024-06-02T15:32:39.350+0000 7f0dbf1d1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:32:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:39 smithi090 ceph-mon[31595]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:32:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:39 smithi090 ceph-mon[31595]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T15:32:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:39 smithi090 ceph-mon[31595]: from='osd.2 ' entity='osd.2' 2024-06-02T15:32:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:40.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:39 smithi120 ceph-mon[36451]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:32:40.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:39 smithi120 ceph-mon[36451]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T15:32:40.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:40.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:39 smithi120 ceph-mon[36451]: from='osd.2 ' entity='osd.2' 2024-06-02T15:32:40.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:40.212 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T15:32:40.213 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T15:32:40.213 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-06-02T15:32:40.213 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-06-02T15:32:40.213 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0352719 s, 297 MB/s 2024-06-02T15:32:40.213 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-06-02T15:32:40.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi135 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] boot 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi135 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] boot 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:41.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi135 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi135 to 17088M 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: osd.2 [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] boot 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:32:41.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:41.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-06-02T15:32:41.500 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:32:42.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:42 smithi090 ceph-mon[31595]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:42.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:42 smithi090 ceph-mon[31595]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T15:32:42.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:43.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:42 smithi120 ceph-mon[36451]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:43.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:42 smithi120 ceph-mon[36451]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T15:32:43.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:43.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:42 smithi135 ceph-mon[36839]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:43.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:42 smithi135 ceph-mon[36839]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T15:32:43.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.671+0000 7f2a28600700 1 -- 172.21.15.90:0/3319294331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20104780 msgr2=0x7f2a20106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.671+0000 7f2a28600700 1 --2- 172.21.15.90:0/3319294331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20104780 0x7f2a20106c10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009990 tx=0x7f2a0c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 -- 172.21.15.90:0/3319294331 shutdown_connections 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 --2- 172.21.15.90:0/3319294331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20107150 0x7f2a201075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 --2- 172.21.15.90:0/3319294331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20101e10 0x7f2a20104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 --2- 172.21.15.90:0/3319294331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20104780 0x7f2a20106c10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 -- 172.21.15.90:0/3319294331 >> 172.21.15.90:0/3319294331 conn(0x7f2a200fda30 msgr2=0x7f2a200ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:32:43.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 -- 172.21.15.90:0/3319294331 shutdown_connections 2024-06-02T15:32:43.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.672+0000 7f2a28600700 1 -- 172.21.15.90:0/3319294331 wait complete. 2024-06-02T15:32:43.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 Processor -- start 2024-06-02T15:32:43.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 -- start start 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 0x7f2a201a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 0x7f2a201a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201155b0 con 0x7f2a20107150 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20115430 con 0x7f2a20101e10 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.673+0000 7f2a28600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a20115730 con 0x7f2a20104780 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 0x7f2a201a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33120/0 (socket says 172.21.15.90:33120) 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 -- 172.21.15.90:0/886869912 learned_addr learned my addr 172.21.15.90:0/886869912 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:32:43.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a26600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 0x7f2a201a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 -- 172.21.15.90:0/886869912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 msgr2=0x7f2a201a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 0x7f2a201a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 -- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 msgr2=0x7f2a201a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 0x7f2a201a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 -- 172.21.15.90:0/886869912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201a7860 con 0x7f2a20101e10 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25200700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 0x7f2a201a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a26600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 0x7f2a201a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.674+0000 7f2a25c00700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e9a0 tx=0x7f2a1400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.675+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400cbd0 con 0x7f2a20101e10 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.675+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a1400cd70 con 0x7f2a20101e10 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.675+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a201a7b50 con 0x7f2a20101e10 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.675+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14018680 con 0x7f2a20101e10 2024-06-02T15:32:43.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.675+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201a80e0 con 0x7f2a20101e10 2024-06-02T15:32:43.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.676+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2006a230 con 0x7f2a20101e10 2024-06-02T15:32:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.678+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2a14010430 con 0x7f2a20101e10 2024-06-02T15:32:43.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.680+0000 7f2a1e000700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 0x7f2a080b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:32:43.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.680+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f2a140e1470 con 0x7f2a20101e10 2024-06-02T15:32:43.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.680+0000 7f2a25200700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 0x7f2a080b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:32:43.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.681+0000 7f2a25200700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 0x7f2a080b4d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b2e0 tx=0x7f2a0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:32:43.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.684+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a140aa5a0 con 0x7f2a20101e10 2024-06-02T15:32:43.744 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50471]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:43.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:43.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:43 smithi090 ceph-mon[31595]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T15:32:43.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:43.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:43.943+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2a20063980 con 0x7f2a080b2870 2024-06-02T15:32:43.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:43.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:43 smithi135 ceph-mon[36839]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T15:32:43.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:44.024 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50471]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.024 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50471]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.024 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50471]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.025 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50474]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.025 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50474]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.025 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50474]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.025 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:32:43 smithi090 sudo[50474]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:44.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:43 smithi120 ceph-mon[36451]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T15:32:44.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:32:44.058 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44978]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:44.058 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44978]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44978]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44981]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.059 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:32:43 smithi120 sudo[44981]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45564]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:44.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45564]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45564]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45564]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45567]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45567]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45567]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 sudo[45567]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45558]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45558]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45561]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45561]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.318 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45561]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.319 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:32:43 smithi135 sudo[45561]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50495]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:44.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50495]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50495]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50495]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50517]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 sudo[50517]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44984]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44984]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T15:32:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44987]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T15:32:44.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 sudo[44987]: pam_unix(sudo:session): session closed for user root 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:45.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:44 smithi090 ceph-mon[31595]: osdmap e22: 3 total, 3 up, 3 in 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.060 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:44 smithi120 ceph-mon[36451]: osdmap e22: 3 total, 3 up, 3 in 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T15:32:45.069 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:44 smithi135 ceph-mon[36839]: osdmap e22: 3 total, 3 up, 3 in 2024-06-02T15:32:45.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:32:45.680+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2a140a6130 con 0x7f2a20101e10 2024-06-02T15:32:46.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:45 smithi090 ceph-mon[31595]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:32:46.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:45 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:32:46.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:45 smithi090 ceph-mon[31595]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T15:32:46.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:45 smithi120 ceph-mon[36451]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:32:46.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:45 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:32:46.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:45 smithi120 ceph-mon[36451]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T15:32:46.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:45 smithi135 ceph-mon[36839]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:32:46.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:45 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:32:46.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:45 smithi135 ceph-mon[36839]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T15:32:47.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:46 smithi120 ceph-mon[36451]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:47.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:46 smithi135 ceph-mon[36839]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:47.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:46 smithi090 ceph-mon[31595]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:48 smithi090 ceph-mon[31595]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:48 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]: dispatch 2024-06-02T15:32:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:48 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]': finished 2024-06-02T15:32:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:48 smithi090 ceph-mon[31595]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T15:32:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:32:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:48 smithi120 ceph-mon[36451]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:48 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]: dispatch 2024-06-02T15:32:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:48 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]': finished 2024-06-02T15:32:49.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:48 smithi120 ceph-mon[36451]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T15:32:49.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:32:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:48 smithi135 ceph-mon[36839]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:48 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]: dispatch 2024-06-02T15:32:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:48 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4269233877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "276c9e04-fa0c-4005-aef6-72afffc49bab"}]': finished 2024-06-02T15:32:49.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:48 smithi135 ceph-mon[36839]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T15:32:49.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:32:50.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:49 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3754015686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:50.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:49 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3754015686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:50.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:49 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3754015686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:32:51.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:50 smithi090 ceph-mon[31595]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:51.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:50 smithi120 ceph-mon[36451]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:50 smithi135 ceph-mon[36839]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:52.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:52 smithi090 ceph-mon[31595]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:53.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:52 smithi120 ceph-mon[36451]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:53.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:52 smithi135 ceph-mon[36839]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:55.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:54 smithi120 ceph-mon[36451]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:55.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:54 smithi135 ceph-mon[36839]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:55.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:54 smithi090 ceph-mon[31595]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:57.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:56 smithi090 ceph-mon[31595]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:57.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:56 smithi120 ceph-mon[36451]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:57.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:56 smithi135 ceph-mon[36839]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T15:32:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:57 smithi120 ceph-mon[36451]: Deploying daemon osd.3 on smithi090 2024-06-02T15:32:58.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T15:32:58.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:58.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:57 smithi135 ceph-mon[36839]: Deploying daemon osd.3 on smithi090 2024-06-02T15:32:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T15:32:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:32:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:57 smithi090 ceph-mon[31595]: Deploying daemon osd.3 on smithi090 2024-06-02T15:32:59.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:32:58 smithi090 ceph-mon[31595]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:32:58 smithi120 ceph-mon[36451]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:32:59.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:32:58 smithi135 ceph-mon[36839]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:00 smithi120 ceph-mon[36451]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:00 smithi135 ceph-mon[36839]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:01.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:00 smithi090 ceph-mon[31595]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:02 smithi120 ceph-mon[36451]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:02 smithi135 ceph-mon[36839]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:03.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:02 smithi090 ceph-mon[31595]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:05.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:04 smithi120 ceph-mon[36451]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:04 smithi135 ceph-mon[36839]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:05.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:04 smithi090 ceph-mon[31595]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:06.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:06 smithi090 ceph-mon[31595]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:07.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:06 smithi120 ceph-mon[36451]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:07.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:07.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:07.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:07.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:06 smithi135 ceph-mon[36839]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:07.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:07.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:07.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:08.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:08 smithi090 ceph-mon[31595]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:09.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:08 smithi120 ceph-mon[36451]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:09.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:08 smithi135 ceph-mon[36839]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:10.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:10 smithi090 ceph-mon[31595]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:10.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.541+0000 7f2a1e000700 1 -- 172.21.15.90:0/886869912 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a20063980 con 0x7f2a080b2870 2024-06-02T15:33:10.540 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-06-02T15:33:10.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.544+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 msgr2=0x7f2a080b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.544+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 0x7f2a080b4d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b2e0 tx=0x7f2a0c005fb0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.544+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 msgr2=0x7f2a2019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.544+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e9a0 tx=0x7f2a1400ecb0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 shutdown_connections 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2a080b2870 0x7f2a080b4d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a20107150 0x7f2a201a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a20101e10 0x7f2a2019fc10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 --2- 172.21.15.90:0/886869912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a20104780 0x7f2a201a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 >> 172.21.15.90:0/886869912 conn(0x7f2a200fda30 msgr2=0x7f2a20102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 shutdown_connections 2024-06-02T15:33:10.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:33:10.545+0000 7f2a28600700 1 -- 172.21.15.90:0/886869912 wait complete. 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:10 smithi120 ceph-mon[36451]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:10 smithi135 ceph-mon[36839]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T15:33:11.363 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.3.service 2024-06-02T15:33:11.367 INFO:tasks.cephadm:Deploying osd.4 on smithi120 with /dev/vg_nvme/lv_3... 2024-06-02T15:33:11.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T15:33:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:11.611 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:33:12.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:12 smithi120 ceph-mon[36451]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:33:12.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:12 smithi120 ceph-mon[36451]: osdmap e25: 4 total, 3 up, 4 in 2024-06-02T15:33:12.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:12 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:12 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:12 smithi120 ceph-mon[36451]: osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] boot 2024-06-02T15:33:12.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:12 smithi135 ceph-mon[36839]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:33:12.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:12 smithi135 ceph-mon[36839]: osdmap e25: 4 total, 3 up, 4 in 2024-06-02T15:33:12.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:12 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:12 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:12 smithi135 ceph-mon[36839]: osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] boot 2024-06-02T15:33:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:12 smithi090 ceph-mon[31595]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:33:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:12 smithi090 ceph-mon[31595]: osdmap e25: 4 total, 3 up, 4 in 2024-06-02T15:33:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:12 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:12 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:12 smithi090 ceph-mon[31595]: osd.3 [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] boot 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111962 s, 937 MB/s 2024-06-02T15:33:13.437 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-02T15:33:13.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:33:13.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:33:13.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:13.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T15:33:13.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:13.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:13 smithi090 ceph-mon[31595]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:13.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:13 smithi120 ceph-mon[36451]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T15:33:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:13 smithi135 ceph-mon[36839]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T15:33:14.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_3 2024-06-02T15:33:14.387 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:33:15.431 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:15 smithi120 ceph-mon[36451]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:15.431 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:15 smithi120 ceph-mon[36451]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T15:33:15.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:15 smithi135 ceph-mon[36839]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:15.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:15 smithi135 ceph-mon[36839]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T15:33:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:15 smithi090 ceph-mon[31595]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:15.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:15 smithi090 ceph-mon[31595]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 -- 172.21.15.120:0/3801892107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1047a0 msgr2=0x7fc62c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 --2- 172.21.15.120:0/3801892107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1047a0 0x7fc62c104c00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0099c0 tx=0x7fc61c009cd0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 -- 172.21.15.120:0/3801892107 shutdown_connections 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 --2- 172.21.15.120:0/3801892107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc62c105140 0x7fc62c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 --2- 172.21.15.120:0/3801892107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62c1035a0 0x7fc62c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 --2- 172.21.15.120:0/3801892107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1047a0 0x7fc62c104c00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 -- 172.21.15.120:0/3801892107 >> 172.21.15.120:0/3801892107 conn(0x7fc62c0feb40 msgr2=0x7fc62c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 -- 172.21.15.120:0/3801892107 shutdown_connections 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.987+0000 7fc634051700 1 -- 172.21.15.120:0/3801892107 wait complete. 2024-06-02T15:33:15.994 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 Processor -- start 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 -- start start 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62c1047a0 0x7fc62c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc62c105140 0x7fc62c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10f130 con 0x7fc62c1047a0 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10efb0 con 0x7fc62c1035a0 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c10f2b0 con 0x7fc62c105140 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:36446/0 (socket says 172.21.15.120:36446) 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 -- 172.21.15.120:0/892593341 learned_addr learned my addr 172.21.15.120:0/892593341 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 -- 172.21.15.120:0/892593341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc62c105140 msgr2=0x7fc62c1a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:15.995 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc62c105140 0x7fc62c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 -- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62c1047a0 msgr2=0x7fc62c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62c1047a0 0x7fc62c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 -- 172.21.15.120:0/892593341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c1a14f0 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc631ded700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc62800b500 tx=0x7fc62800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628015020 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c1a17e0 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc628011ce0 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.988+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc628010d50 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.989+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1a1df0 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.989+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c06a230 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.991+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc628018020 con 0x7fc62c1035a0 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.991+0000 7fc622ffd700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 0x7fc6180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:15.996 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.991+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fc6280e0dc0 con 0x7fc62c1035a0 2024-06-02T15:33:15.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.992+0000 7fc6315ec700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 0x7fc6180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:33:15.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.993+0000 7fc6315ec700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 0x7fc6180b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098e0 tx=0x7fc61c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:33:15.997 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:15.996+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc62801f020 con 0x7fc62c1035a0 2024-06-02T15:33:16.203 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:16.205+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc62c063980 con 0x7fc6180b2880 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi090 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 8544M 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:17.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:17 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi090 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 8544M 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:17.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:17 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi090 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 8544M 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:17 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:19.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:33:19.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/1438617775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]': finished 2024-06-02T15:33:19.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: osdmap e29: 5 total, 4 up, 5 in 2024-06-02T15:33:19.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/1438617775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]': finished 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: osdmap e29: 5 total, 4 up, 5 in 2024-06-02T15:33:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:33:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/1438617775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]: dispatch 2024-06-02T15:33:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62594a89-8539-441e-9ae7-fa5938668b30"}]': finished 2024-06-02T15:33:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: osdmap e29: 5 total, 4 up, 5 in 2024-06-02T15:33:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:20.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/469884044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:20.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/469884044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/469884044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:21.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:21 smithi120 ceph-mon[36451]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:33:21.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:21 smithi135 ceph-mon[36839]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:33:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:21 smithi090 ceph-mon[31595]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:33:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:23 smithi090 ceph-mon[31595]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:33:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:23 smithi120 ceph-mon[36451]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:33:23.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:23 smithi135 ceph-mon[36839]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:33:25.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:25 smithi120 ceph-mon[36451]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:33:25.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:25 smithi135 ceph-mon[36839]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:33:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:25 smithi090 ceph-mon[31595]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:33:27.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:27 smithi120 ceph-mon[36451]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:33:27.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T15:33:27.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:27.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:27 smithi135 ceph-mon[36839]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:33:27.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T15:33:27.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:27 smithi090 ceph-mon[31595]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:33:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T15:33:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:28.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:28 smithi090 ceph-mon[31595]: Deploying daemon osd.4 on smithi120 2024-06-02T15:33:28.551 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:28 smithi120 ceph-mon[36451]: Deploying daemon osd.4 on smithi120 2024-06-02T15:33:28.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:28 smithi135 ceph-mon[36839]: Deploying daemon osd.4 on smithi120 2024-06-02T15:33:29.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:29 smithi090 ceph-mon[31595]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:29 smithi120 ceph-mon[36451]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:29 smithi135 ceph-mon[36839]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:31.351 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:31 smithi120 ceph-mon[36451]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:31.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:31 smithi135 ceph-mon[36839]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:31 smithi090 ceph-mon[31595]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:33.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:32 smithi120 ceph-mon[36451]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:33.311 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.311 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.311 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:32 smithi135 ceph-mon[36839]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:32 smithi090 ceph-mon[31595]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:35.028 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:34 smithi120 ceph-mon[36451]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:35.089 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.091+0000 7fc622ffd700 1 -- 172.21.15.120:0/892593341 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc62c063980 con 0x7fc6180b2880 2024-06-02T15:33:35.090 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 4 on host 'smithi120' 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 msgr2=0x7fc6180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 0x7fc6180b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098e0 tx=0x7fc61c000bc0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 msgr2=0x7fc62c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc62800b500 tx=0x7fc62800b8c0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 shutdown_connections 2024-06-02T15:33:35.092 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc6180b2880 0x7fc6180b4d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc62c105140 0x7fc62c1a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc62c1035a0 0x7fc62c199890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 --2- 172.21.15.120:0/892593341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc62c1047a0 0x7fc62c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 >> 172.21.15.120:0/892593341 conn(0x7fc62c0feb40 msgr2=0x7fc62c107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 shutdown_connections 2024-06-02T15:33:35.093 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:33:35.094+0000 7fc634051700 1 -- 172.21.15.120:0/892593341 wait complete. 2024-06-02T15:33:35.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:34 smithi135 ceph-mon[36839]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:34 smithi090 ceph-mon[31595]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:36.102 DEBUG:teuthology.orchestra.run.smithi120:osd.4> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.4.service 2024-06-02T15:33:36.105 INFO:tasks.cephadm:Deploying osd.5 on smithi135 with /dev/vg_nvme/lv_3... 2024-06-02T15:33:36.105 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T15:33:36.352 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:33:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:36 smithi090 ceph-mon[31595]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:36 smithi090 ceph-mon[31595]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:36.474 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.474 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.474 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:36 smithi135 ceph-mon[36839]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:36.474 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:36 smithi135 ceph-mon[36839]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:36 smithi120 ceph-mon[36451]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:36 smithi120 ceph-mon[36451]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:37.309 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:33:37.104+0000 7fcb88e34700 -1 osd.4 0 waiting for initial osdmap 2024-06-02T15:33:37.309 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:33:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:33:37.110+0000 7fcb84c27700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:37.500 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:33:37.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:37.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:38.340 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:38 smithi135 ceph-mon[36839]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:33:38.340 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:38 smithi135 ceph-mon[36839]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T15:33:38.341 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:38.341 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:38 smithi090 ceph-mon[31595]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:33:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:38 smithi090 ceph-mon[31595]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T15:33:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131817 s, 795 MB/s 2024-06-02T15:33:38.496 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-02T15:33:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:38 smithi120 ceph-mon[36451]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:33:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:38 smithi120 ceph-mon[36451]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T15:33:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:39.131 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] boot 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: osdmap e32: 5 total, 5 up, 5 in 2024-06-02T15:33:39.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:39.373 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:33:39.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:33:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:33:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] boot 2024-06-02T15:33:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: osdmap e32: 5 total, 5 up, 5 in 2024-06-02T15:33:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: osd.4 [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] boot 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: osdmap e32: 5 total, 5 up, 5 in 2024-06-02T15:33:39.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T15:33:40.478 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T15:33:40.478 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T15:33:40.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:33:40.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:41.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 -- 172.21.15.135:0/487564333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c1030e0 msgr2=0x7f314c10a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:41.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 --2- 172.21.15.135:0/487564333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c1030e0 0x7f314c10a980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f314400b0a0 tx=0x7f314400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 -- 172.21.15.135:0/487564333 shutdown_connections 2024-06-02T15:33:41.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 --2- 172.21.15.135:0/487564333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c1030e0 0x7f314c10a980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 --2- 172.21.15.135:0/487564333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f314c103ac0 0x7f314c103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 --2- 172.21.15.135:0/487564333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 0x7f314c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 -- 172.21.15.135:0/487564333 >> 172.21.15.135:0/487564333 conn(0x7f314c0fd1b0 msgr2=0x7f314c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 -- 172.21.15.135:0/487564333 shutdown_connections 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.026+0000 7f31517ce700 1 -- 172.21.15.135:0/487564333 wait complete. 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 Processor -- start 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 -- start start 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 0x7f314c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f314c1030e0 0x7f314c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:41.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10e030 con 0x7f314c1030e0 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10deb0 con 0x7f314c102720 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f31517ce700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10e1b0 con 0x7f314c103ac0 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.027+0000 7f314b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39968/0 (socket says 172.21.15.135:39968) 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 -- 172.21.15.135:0/368650261 learned_addr learned my addr 172.21.15.135:0/368650261 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314affd700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 0x7f314c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 -- 172.21.15.135:0/368650261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 msgr2=0x7f314c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 0x7f314c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 -- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f314c1030e0 msgr2=0x7f314c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f314c1030e0 0x7f314c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 -- 172.21.15.135:0/368650261 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c1a0310 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f314b7fe700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f314400cf00 tx=0x7f314400cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144021070 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c1a05a0 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f314401d900 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144005350 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.028+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1a0bb0 con 0x7f314c103ac0 2024-06-02T15:33:41.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.029+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c06a230 con 0x7f314c103ac0 2024-06-02T15:33:41.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.030+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3144024070 con 0x7f314c103ac0 2024-06-02T15:33:41.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.031+0000 7f3133fff700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 0x7f31340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:33:41.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.031+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f314400f020 con 0x7f314c103ac0 2024-06-02T15:33:41.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.032+0000 7f314affd700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 0x7f31340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:33:41.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.033+0000 7f314affd700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 0x7f31340b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f314c06bad0 tx=0x7f313c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:33:41.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.035+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f314402b020 con 0x7f314c103ac0 2024-06-02T15:33:41.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:33:41.247+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f314c063980 con 0x7f31340b2880 2024-06-02T15:33:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:41 smithi120 ceph-mon[36451]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:41 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi120 2024-06-02T15:33:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:41 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-02T15:33:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:41 smithi135 ceph-mon[36839]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:41 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi120 2024-06-02T15:33:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:41 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-02T15:33:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:41 smithi090 ceph-mon[31595]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:41 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi120 2024-06-02T15:33:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:41 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 8544M 2024-06-02T15:33:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:42 smithi120 ceph-mon[36451]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:42 smithi135 ceph-mon[36839]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:42.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:42 smithi090 ceph-mon[31595]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:33:42.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:33:42.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:33:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:43.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:43 smithi120 ceph-mon[36451]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:43 smithi135 ceph-mon[36839]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:43 smithi090 ceph-mon[31595]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:44 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/310582919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:44 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:44 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]': finished 2024-06-02T15:33:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:44 smithi120 ceph-mon[36451]: osdmap e34: 6 total, 5 up, 6 in 2024-06-02T15:33:44.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:33:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:44 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/310582919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:44 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:44 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]': finished 2024-06-02T15:33:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:44 smithi135 ceph-mon[36839]: osdmap e34: 6 total, 5 up, 6 in 2024-06-02T15:33:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:33:44.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/310582919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:44 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]: dispatch 2024-06-02T15:33:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:44 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd04176c-ed63-4796-99b1-720481048d12"}]': finished 2024-06-02T15:33:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:44 smithi090 ceph-mon[31595]: osdmap e34: 6 total, 5 up, 6 in 2024-06-02T15:33:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:33:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:45 smithi120 ceph-mon[36451]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:45 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/34910108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:45.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:45 smithi135 ceph-mon[36839]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:45.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:45 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/34910108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:45 smithi090 ceph-mon[31595]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:45 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/34910108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:33:47.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:47 smithi120 ceph-mon[36451]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:47.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:47 smithi135 ceph-mon[36839]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:47.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:47 smithi090 ceph-mon[31595]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:49.222 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:49 smithi135 ceph-mon[36839]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:49 smithi120 ceph-mon[36451]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:49.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:49 smithi090 ceph-mon[31595]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:51.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:51 smithi135 ceph-mon[36839]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:51 smithi120 ceph-mon[36451]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:51.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:51 smithi090 ceph-mon[31595]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:52 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T15:33:52.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:52 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:52.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:52 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T15:33:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:52 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:52.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:52 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T15:33:52.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:52 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:33:53.314 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:53 smithi135 ceph-mon[36839]: Deploying daemon osd.5 on smithi135 2024-06-02T15:33:53.314 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:53 smithi135 ceph-mon[36839]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:53 smithi120 ceph-mon[36451]: Deploying daemon osd.5 on smithi135 2024-06-02T15:33:53.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:53 smithi120 ceph-mon[36451]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:53 smithi090 ceph-mon[31595]: Deploying daemon osd.5 on smithi135 2024-06-02T15:33:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:53 smithi090 ceph-mon[31595]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:55.366 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:55 smithi135 ceph-mon[36839]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:55 smithi120 ceph-mon[36451]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:55 smithi090 ceph-mon[31595]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:57.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:57 smithi120 ceph-mon[36451]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:57.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:57 smithi135 ceph-mon[36839]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:57 smithi090 ceph-mon[31595]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:33:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:33:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:33:59 smithi135 ceph-mon[36839]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:33:59 smithi120 ceph-mon[36451]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:33:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:33:59 smithi090 ceph-mon[31595]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:00.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.885+0000 7f3133fff700 1 -- 172.21.15.135:0/368650261 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f314c063980 con 0x7f31340b2880 2024-06-02T15:34:00.884 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 5 on host 'smithi135' 2024-06-02T15:34:00.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 msgr2=0x7f31340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 0x7f31340b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f314c06bad0 tx=0x7f313c009190 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 msgr2=0x7f314c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f314400cf00 tx=0x7f314400cf30 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 shutdown_connections 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31340b2880 0x7f31340b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f314c103ac0 0x7f314c19fc70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f314c102720 0x7f314c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 --2- 172.21.15.135:0/368650261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f314c1030e0 0x7f314c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.888+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 >> 172.21.15.135:0/368650261 conn(0x7f314c0fd1b0 msgr2=0x7f314c106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.889+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 shutdown_connections 2024-06-02T15:34:00.887 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:34:00.889+0000 7f31517ce700 1 -- 172.21.15.135:0/368650261 wait complete. 2024-06-02T15:34:01.263 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.264 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:00 smithi135 ceph-mon[36839]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:01.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:00 smithi120 ceph-mon[36451]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:00 smithi090 ceph-mon[31595]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T15:34:01.589 DEBUG:teuthology.orchestra.run.smithi135:osd.5> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.5.service 2024-06-02T15:34:01.591 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-06-02T15:34:01.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T15:34:02.027 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:34:03.035 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:03.035 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T15:34:03.035 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T15:34:03.035 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:03.036 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:03.036 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-mon[36839]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:03.036 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:34:02.884+0000 7f8484155700 -1 osd.5 0 waiting for initial osdmap 2024-06-02T15:34:03.036 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:34:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:34:02.893+0000 7f847ff48700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:34:03.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:03.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T15:34:03.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T15:34:03.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:03.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:03.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:02 smithi090 ceph-mon[31595]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:03.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T15:34:03.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T15:34:03.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: from='osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:03.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:03.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:02 smithi120 ceph-mon[36451]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:34:04.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:03 smithi135 ceph-mon[36839]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:34:04.158 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:03 smithi135 ceph-mon[36839]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T15:34:04.158 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:04.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:03 smithi090 ceph-mon[31595]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:34:04.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:03 smithi090 ceph-mon[31595]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T15:34:04.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:04.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:03 smithi120 ceph-mon[36451]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:34:04.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:03 smithi120 ceph-mon[36451]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T15:34:04.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0361132 s, 290 MB/s 2024-06-02T15:34:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-06-02T15:34:05.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] boot 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: osdmap e37: 6 total, 6 up, 6 in 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:05.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:04 smithi090 ceph-mon[31595]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] boot 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: osdmap e37: 6 total, 6 up, 6 in 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:05.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:05.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:04 smithi120 ceph-mon[36451]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: osd.5 [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] boot 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: osdmap e37: 6 total, 6 up, 6 in 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:05.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:05.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:05.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:04 smithi135 ceph-mon[36839]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T15:34:05.988 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-06-02T15:34:06.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:05 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi135 2024-06-02T15:34:06.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:05 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-02T15:34:06.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:05 smithi120 ceph-mon[36451]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T15:34:06.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:05 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi135 2024-06-02T15:34:06.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:05 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-02T15:34:06.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:05 smithi135 ceph-mon[36839]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T15:34:06.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:05 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi135 2024-06-02T15:34:06.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:05 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi135 to 8544M 2024-06-02T15:34:06.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:05 smithi090 ceph-mon[31595]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T15:34:06.413 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:34:07.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:06 smithi120 ceph-mon[36451]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:07.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:06 smithi135 ceph-mon[36839]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:07.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:06 smithi090 ceph-mon[31595]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:08.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.738+0000 7f9ebf200700 1 -- 172.21.15.90:0/3208047421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb81069e0 msgr2=0x7f9eb8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:08.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.738+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3208047421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb81069e0 0x7f9eb8106e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40099c0 tx=0x7f9ea4009cd0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 -- 172.21.15.90:0/3208047421 shutdown_connections 2024-06-02T15:34:08.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3208047421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb8107380 0x7f9eb810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3208047421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb8105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3208047421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb81069e0 0x7f9eb8106e40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 -- 172.21.15.90:0/3208047421 >> 172.21.15.90:0/3208047421 conn(0x7f9eb8100da0 msgr2=0x7f9eb81031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:34:08.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 -- 172.21.15.90:0/3208047421 shutdown_connections 2024-06-02T15:34:08.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.739+0000 7f9ebf200700 1 -- 172.21.15.90:0/3208047421 wait complete. 2024-06-02T15:34:08.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 Processor -- start 2024-06-02T15:34:08.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 -- start start 2024-06-02T15:34:08.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:08.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb81069e0 0x7f9eb819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:08.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb81112d0 con 0x7f9eb8107380 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8111150 con 0x7f9eb81057e0 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.740+0000 7f9ebf200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8111450 con 0x7f9eb81069e0 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58636/0 (socket says 172.21.15.90:58636) 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 -- 172.21.15.90:0/3447993703 learned_addr learned my addr 172.21.15.90:0/3447993703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9eb7e00700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9eb7400700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb81069e0 0x7f9eb819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 -- 172.21.15.90:0/3447993703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb81069e0 msgr2=0x7f9eb819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb81069e0 0x7f9eb819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 -- 172.21.15.90:0/3447993703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 msgr2=0x7f9eb819b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb819b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:08.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9ebd200700 1 -- 172.21.15.90:0/3447993703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb81a3550 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.741+0000 7f9eb7e00700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9ebd200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000ef60 tx=0x7f9eb000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00092e0 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eb0004bb0 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0004020 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb81a37e0 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.742+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb81a3cd0 con 0x7f9eb8107380 2024-06-02T15:34:08.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.743+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb806a230 con 0x7f9eb8107380 2024-06-02T15:34:08.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.745+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9eb001b030 con 0x7f9eb8107380 2024-06-02T15:34:08.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.747+0000 7f9eb4c00700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 0x7f9e9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:08.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.747+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f9eb00e0dc0 con 0x7f9eb8107380 2024-06-02T15:34:08.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.747+0000 7f9eb7e00700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 0x7f9e9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:08.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.748+0000 7f9eb7e00700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 0x7f9e9c0b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8106840 tx=0x7f9ea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:34:08.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.752+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9eb0022020 con 0x7f9eb8107380 2024-06-02T15:34:08.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:08.999+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9eb8063980 con 0x7f9e9c0b2880 2024-06-02T15:34:09.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:08 smithi120 ceph-mon[36451]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-02T15:34:09.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:08 smithi135 ceph-mon[36839]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-02T15:34:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:08 smithi090 ceph-mon[31595]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-06-02T15:34:10.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:09 smithi090 ceph-mon[31595]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:10.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:10.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:10.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:10.218 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:09 smithi135 ceph-mon[36839]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:10.218 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:10.218 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:10.218 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:10.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:09 smithi120 ceph-mon[36451]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:10.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:10.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:10.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:11.230 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:10 smithi135 ceph-mon[36839]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:34:11.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:10 smithi120 ceph-mon[36451]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:34:11.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:10 smithi090 ceph-mon[31595]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:34:13.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:12 smithi120 ceph-mon[36451]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T15:34:13.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:12 smithi135 ceph-mon[36839]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T15:34:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:12 smithi090 ceph-mon[31595]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T15:34:14.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]: dispatch 2024-06-02T15:34:14.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]': finished 2024-06-02T15:34:14.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:13 smithi090 ceph-mon[31595]: osdmap e40: 7 total, 6 up, 7 in 2024-06-02T15:34:14.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:14.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:13 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]: dispatch 2024-06-02T15:34:14.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:13 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]': finished 2024-06-02T15:34:14.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:13 smithi120 ceph-mon[36451]: osdmap e40: 7 total, 6 up, 7 in 2024-06-02T15:34:14.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:14.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:13 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]: dispatch 2024-06-02T15:34:14.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:13 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1562537247' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4ffd454-114d-46e2-9735-49849a2fa18b"}]': finished 2024-06-02T15:34:14.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:13 smithi135 ceph-mon[36839]: osdmap e40: 7 total, 6 up, 7 in 2024-06-02T15:34:14.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:15.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:14 smithi120 ceph-mon[36451]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:34:15.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:14 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2348466238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:15.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:14 smithi135 ceph-mon[36839]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:34:15.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:14 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2348466238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:15.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:14 smithi090 ceph-mon[31595]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:34:15.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:14 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2348466238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:17.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:16 smithi120 ceph-mon[36451]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-02T15:34:17.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:16 smithi135 ceph-mon[36839]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-02T15:34:17.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:16 smithi090 ceph-mon[31595]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-06-02T15:34:19.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:18 smithi120 ceph-mon[36451]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:19.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:18 smithi135 ceph-mon[36839]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:18 smithi090 ceph-mon[31595]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:21.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:20 smithi090 ceph-mon[31595]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:21.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:20 smithi120 ceph-mon[36451]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:21.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:20 smithi135 ceph-mon[36839]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:23.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:22 smithi120 ceph-mon[36451]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:23.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T15:34:23.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:22 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:22 smithi135 ceph-mon[36839]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T15:34:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:22 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:22 smithi090 ceph-mon[31595]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T15:34:23.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:22 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:24.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:23 smithi090 ceph-mon[31595]: Deploying daemon osd.6 on smithi090 2024-06-02T15:34:24.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:23 smithi120 ceph-mon[36451]: Deploying daemon osd.6 on smithi090 2024-06-02T15:34:24.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:23 smithi135 ceph-mon[36839]: Deploying daemon osd.6 on smithi090 2024-06-02T15:34:25.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:24 smithi120 ceph-mon[36451]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:25.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:24 smithi135 ceph-mon[36839]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:25.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:24 smithi090 ceph-mon[31595]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:27.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:26 smithi120 ceph-mon[36451]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:26 smithi135 ceph-mon[36839]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:26 smithi090 ceph-mon[31595]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:28 smithi090 ceph-mon[31595]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:28 smithi120 ceph-mon[36451]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:28 smithi135 ceph-mon[36839]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:31.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:30 smithi090 ceph-mon[31595]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:31.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:30 smithi120 ceph-mon[36451]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:31.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:30 smithi135 ceph-mon[36839]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:32.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:32.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:32 smithi120 ceph-mon[36451]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:32.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:32.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:32 smithi090 ceph-mon[31595]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:32.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:32.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:32.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:32 smithi135 ceph-mon[36839]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:34.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:34 smithi090 ceph-mon[31595]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:35.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:34 smithi120 ceph-mon[36451]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:35.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:34 smithi135 ceph-mon[36839]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:36.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.437+0000 7f9eb4c00700 1 -- 172.21.15.90:0/3447993703 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9eb8063980 con 0x7f9e9c0b2880 2024-06-02T15:34:36.436 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 msgr2=0x7f9e9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 0x7f9e9c0b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8106840 tx=0x7f9ea8009260 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 msgr2=0x7f9eb81a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000ef60 tx=0x7f9eb000ef90 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 shutdown_connections 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9e9c0b2880 0x7f9e9c0b4d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eb8107380 0x7f9eb81a2eb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9eb81057e0 0x7f9eb819b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 --2- 172.21.15.90:0/3447993703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eb81069e0 0x7f9eb819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 >> 172.21.15.90:0/3447993703 conn(0x7f9eb8100da0 msgr2=0x7f9eb8109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:34:36.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 shutdown_connections 2024-06-02T15:34:36.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:34:36.441+0000 7f9ebf200700 1 -- 172.21.15.90:0/3447993703 wait complete. 2024-06-02T15:34:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:36 smithi090 ceph-mon[31595]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:36 smithi090 ceph-mon[31595]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:36 smithi090 ceph-mon[31595]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:36.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:36 smithi120 ceph-mon[36451]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:36 smithi120 ceph-mon[36451]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:36 smithi120 ceph-mon[36451]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:36.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:36.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:36 smithi135 ceph-mon[36839]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:36 smithi135 ceph-mon[36839]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T15:34:36.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:36 smithi135 ceph-mon[36839]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:37.219 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.6.service 2024-06-02T15:34:37.223 INFO:tasks.cephadm:Deploying osd.7 on smithi120 with /dev/vg_nvme/lv_2... 2024-06-02T15:34:37.224 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T15:34:37.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: osdmap e41: 7 total, 6 up, 7 in 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:37.467 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:34:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: osdmap e41: 7 total, 6 up, 7 in 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: osdmap e41: 7 total, 6 up, 7 in 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:34:37.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:34:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: osdmap e42: 7 total, 6 up, 7 in 2024-06-02T15:34:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-mon[31595]: from='osd.6 ' entity='osd.6' 2024-06-02T15:34:38.626 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:34:38.291+0000 7fb0d5600700 -1 osd.6 0 waiting for initial osdmap 2024-06-02T15:34:38.627 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:34:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:34:38.318+0000 7fb0cb400700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:34:38.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:34:38.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: osdmap e42: 7 total, 6 up, 7 in 2024-06-02T15:34:38.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.746 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:38.747 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:38 smithi120 ceph-mon[36451]: from='osd.6 ' entity='osd.6' 2024-06-02T15:34:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:34:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: osdmap e42: 7 total, 6 up, 7 in 2024-06-02T15:34:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:38.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-06-02T15:34:38.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:38 smithi135 ceph-mon[36839]: from='osd.6 ' entity='osd.6' 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011623 s, 902 MB/s 2024-06-02T15:34:39.481 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] boot 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: osdmap e43: 7 total, 7 up, 7 in 2024-06-02T15:34:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] boot 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: osdmap e43: 7 total, 7 up, 7 in 2024-06-02T15:34:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:34:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:34:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: osd.6 [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] boot 2024-06-02T15:34:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: osdmap e43: 7 total, 7 up, 7 in 2024-06-02T15:34:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T15:34:40.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_2 2024-06-02T15:34:40.435 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:34:40.715 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:40 smithi120 ceph-mon[36451]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:40.718 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:40 smithi135 ceph-mon[36839]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:40.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:40 smithi090 ceph-mon[31595]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:41 smithi120 ceph-mon[36451]: osdmap e44: 7 total, 7 up, 7 in 2024-06-02T15:34:41.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:41 smithi135 ceph-mon[36839]: osdmap e44: 7 total, 7 up, 7 in 2024-06-02T15:34:41.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:41 smithi090 ceph-mon[31595]: osdmap e44: 7 total, 7 up, 7 in 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 -- 172.21.15.120:0/3662176164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d80104040 msgr2=0x7f8d8010a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 --2- 172.21.15.120:0/3662176164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d80104040 0x7f8d8010a8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800b0d0 tx=0x7f8d7800b3e0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 -- 172.21.15.120:0/3662176164 shutdown_connections 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 --2- 172.21.15.120:0/3662176164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d80104040 0x7f8d8010a8f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 --2- 172.21.15.120:0/3662176164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801024a0 0x7f8d801028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 --2- 172.21.15.120:0/3662176164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d801036a0 0x7f8d80103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 -- 172.21.15.120:0/3662176164 >> 172.21.15.120:0/3662176164 conn(0x7f8d800fda40 msgr2=0x7f8d800ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 -- 172.21.15.120:0/3662176164 shutdown_connections 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.211+0000 7f8d88365700 1 -- 172.21.15.120:0/3662176164 wait complete. 2024-06-02T15:34:42.210 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 Processor -- start 2024-06-02T15:34:42.211 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 -- start start 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801036a0 0x7f8d80198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d80104040 0x7f8d8019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8010dfe0 con 0x7f8d80104040 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8010de60 con 0x7f8d801024a0 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.212+0000 7f8d88365700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8010e160 con 0x7f8d801036a0 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:40754/0 (socket says 172.21.15.120:40754) 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 -- 172.21.15.120:0/2768854324 learned_addr learned my addr 172.21.15.120:0/2768854324 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d85900700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801036a0 0x7f8d80198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86902700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d80104040 0x7f8d8019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:42.213 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 -- 172.21.15.120:0/2768854324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801036a0 msgr2=0x7f8d80198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801036a0 0x7f8d80198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 -- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d80104040 msgr2=0x7f8d8019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d80104040 0x7f8d8019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 -- 172.21.15.120:0/2768854324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d801a0220 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d86101700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c990 tx=0x7f8d7c00cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c007b00 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d7c007ca0 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d801a0510 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c018660 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.213+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801a0ad0 con 0x7f8d801024a0 2024-06-02T15:34:42.214 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.214+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8006a230 con 0x7f8d801024a0 2024-06-02T15:34:42.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.215+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8d7c00f450 con 0x7f8d801024a0 2024-06-02T15:34:42.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.216+0000 7f8d777fe700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 0x7f8d6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:34:42.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.216+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f8d7c0e14e0 con 0x7f8d801024a0 2024-06-02T15:34:42.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.216+0000 7f8d85900700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 0x7f8d6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:34:42.215 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.217+0000 7f8d85900700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 0x7f8d6c0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009a10 tx=0x7f8d70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:34:42.217 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.220+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d7c014030 con 0x7f8d801024a0 2024-06-02T15:34:42.434 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:34:42.436+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8d80063980 con 0x7f8d6c0b2880 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi090 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 5696M 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:43.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:43 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:43.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi090 2024-06-02T15:34:43.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 5696M 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:43 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi090 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 5696M 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:34:43.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:43 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:44.354 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:44 smithi120 ceph-mon[36451]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:44.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:44 smithi090 ceph-mon[31595]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:44.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:44 smithi135 ceph-mon[36839]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:34:45.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:45 smithi090 ceph-mon[31595]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:45 smithi120 ceph-mon[36451]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:45 smithi135 ceph-mon[36839]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:46.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/1526918143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]': finished 2024-06-02T15:34:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: osdmap e45: 8 total, 7 up, 8 in 2024-06-02T15:34:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:34:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:46 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/3405372344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/1526918143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]': finished 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: osdmap e45: 8 total, 7 up, 8 in 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:34:46.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:46 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/3405372344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:46.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/1526918143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]: dispatch 2024-06-02T15:34:46.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a44884e0-5c29-4809-8204-7e00dcbf3285"}]': finished 2024-06-02T15:34:46.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: osdmap e45: 8 total, 7 up, 8 in 2024-06-02T15:34:46.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:34:46.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:46 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/3405372344' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:34:47.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:47 smithi090 ceph-mon[31595]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:47 smithi120 ceph-mon[36451]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:47 smithi135 ceph-mon[36839]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:49.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:49 smithi120 ceph-mon[36451]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:49.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:49 smithi090 ceph-mon[31595]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:49.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:49 smithi135 ceph-mon[36839]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:51.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:51 smithi090 ceph-mon[31595]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:51.406 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:51 smithi120 ceph-mon[36451]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:51.427 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:51 smithi135 ceph-mon[36839]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:53.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:53 smithi120 ceph-mon[36451]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:53.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:53 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T15:34:53.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:53 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:53 smithi090 ceph-mon[31595]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:53 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T15:34:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:53 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:53 smithi135 ceph-mon[36839]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:53 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T15:34:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:53 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:34:54.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:54 smithi120 ceph-mon[36451]: Deploying daemon osd.7 on smithi120 2024-06-02T15:34:54.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:54 smithi135 ceph-mon[36839]: Deploying daemon osd.7 on smithi120 2024-06-02T15:34:54.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:54 smithi090 ceph-mon[31595]: Deploying daemon osd.7 on smithi120 2024-06-02T15:34:55.293 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:55 smithi120 ceph-mon[36451]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:55 smithi135 ceph-mon[36839]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:55 smithi090 ceph-mon[31595]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:57.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:57 smithi120 ceph-mon[36451]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:57 smithi135 ceph-mon[36839]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:57 smithi090 ceph-mon[31595]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:59.398 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:59 smithi120 ceph-mon[36451]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:59.398 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.398 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.398 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:34:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:59.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:59 smithi090 ceph-mon[31595]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:59.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:34:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:34:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:59 smithi135 ceph-mon[36839]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:34:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:34:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:34:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:01.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:01 smithi120 ceph-mon[36451]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:01.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:01 smithi135 ceph-mon[36839]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:01 smithi090 ceph-mon[31595]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:02.415 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.417+0000 7f8d777fe700 1 -- 172.21.15.120:0/2768854324 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d80063980 con 0x7f8d6c0b2880 2024-06-02T15:35:02.415 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 7 on host 'smithi120' 2024-06-02T15:35:02.417 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 msgr2=0x7f8d6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 0x7f8d6c0b4d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009a10 tx=0x7f8d70009190 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 msgr2=0x7f8d801985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c990 tx=0x7f8d7c00cca0 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 shutdown_connections 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d6c0b2880 0x7f8d6c0b4d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d80104040 0x7f8d8019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d801024a0 0x7f8d801985c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 --2- 172.21.15.120:0/2768854324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d801036a0 0x7f8d80198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 >> 172.21.15.120:0/2768854324 conn(0x7f8d800fda40 msgr2=0x7f8d80106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 shutdown_connections 2024-06-02T15:35:02.418 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:35:02.420+0000 7f8d88365700 1 -- 172.21.15.120:0/2768854324 wait complete. 2024-06-02T15:35:03.148 DEBUG:teuthology.orchestra.run.smithi120:osd.7> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.7.service 2024-06-02T15:35:03.150 INFO:tasks.cephadm:Deploying osd.8 on smithi135 with /dev/vg_nvme/lv_2... 2024-06-02T15:35:03.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.398 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:35:03.519 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.520 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: osdmap e46: 8 total, 7 up, 8 in 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.679 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-mon[36451]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:04.680 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:35:04.381+0000 7f243503f700 -1 osd.7 0 waiting for initial osdmap 2024-06-02T15:35:04.680 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:35:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:35:04.390+0000 7f2430e32700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: osdmap e46: 8 total, 7 up, 8 in 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.727 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:04 smithi135 ceph-mon[36839]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:04.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T15:35:04.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: osdmap e46: 8 total, 7 up, 8 in 2024-06-02T15:35:04.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:04.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: from='osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:35:04.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:04 smithi090 ceph-mon[31595]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-06-02T15:35:05.506 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T15:35:05.506 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T15:35:05.506 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-02T15:35:05.507 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-02T15:35:05.507 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115025 s, 912 MB/s 2024-06-02T15:35:05.507 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-02T15:35:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:05 smithi135 ceph-mon[36839]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:35:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:05 smithi135 ceph-mon[36839]: osdmap e47: 8 total, 7 up, 8 in 2024-06-02T15:35:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:05 smithi135 ceph-mon[36839]: osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] boot 2024-06-02T15:35:05.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:05 smithi120 ceph-mon[36451]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:35:05.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:05 smithi120 ceph-mon[36451]: osdmap e47: 8 total, 7 up, 8 in 2024-06-02T15:35:05.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:05 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:05 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:05 smithi120 ceph-mon[36451]: osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] boot 2024-06-02T15:35:05.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:05 smithi090 ceph-mon[31595]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:35:05.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:05 smithi090 ceph-mon[31595]: osdmap e47: 8 total, 7 up, 8 in 2024-06-02T15:35:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:05 smithi090 ceph-mon[31595]: osd.7 [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] boot 2024-06-02T15:35:06.161 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-06-02T15:35:06.402 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: osdmap e48: 8 total, 8 up, 8 in 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: osdmap e49: 8 total, 8 up, 8 in 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi120 2024-06-02T15:35:06.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:06.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:06 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: osdmap e48: 8 total, 8 up, 8 in 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: osdmap e49: 8 total, 8 up, 8 in 2024-06-02T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi120 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:06.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:06 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: osdmap e48: 8 total, 8 up, 8 in 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: osdmap e49: 8 total, 8 up, 8 in 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi120 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 5696M 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:06 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:07.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:07 smithi135 ceph-mon[36839]: osdmap e50: 8 total, 8 up, 8 in 2024-06-02T15:35:07.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:07 smithi120 ceph-mon[36451]: osdmap e50: 8 total, 8 up, 8 in 2024-06-02T15:35:07.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:07.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:07.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:07.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:07.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:07 smithi090 ceph-mon[31595]: osdmap e50: 8 total, 8 up, 8 in 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/2904219488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8103690 msgr2=0x7ff9d8103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/2904219488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8103690 0x7ff9d8103af0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009990 tx=0x7ff9c0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/2904219488 shutdown_connections 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/2904219488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8104030 0x7ff9d810a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/2904219488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9d8102490 0x7ff9d81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/2904219488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8103690 0x7ff9d8103af0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/2904219488 >> 172.21.15.135:0/2904219488 conn(0x7ff9d80fda30 msgr2=0x7ff9d80ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/2904219488 shutdown_connections 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/2904219488 wait complete. 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.110+0000 7ff9dd8bc700 1 Processor -- start 2024-06-02T15:35:08.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 -- start start 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 0x7ff9d8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9d8104030 0x7ff9d819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810e070 con 0x7ff9d8104030 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810def0 con 0x7ff9d8103690 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dd8bc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810e1f0 con 0x7ff9d8102490 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44726/0 (socket says 172.21.15.135:44726) 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 -- 172.21.15.135:0/1478582154 learned_addr learned my addr 172.21.15.135:0/1478582154 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9cffff700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 0x7ff9d8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 -- 172.21.15.135:0/1478582154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 msgr2=0x7ff9d8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:08.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 0x7ff9d8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 -- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9d8104030 msgr2=0x7ff9d819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9d8104030 0x7ff9d819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 -- 172.21.15.135:0/1478582154 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d81a0380 con 0x7ff9d8102490 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9cffff700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 0x7ff9d8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9d6ffd700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b530 tx=0x7ff9c800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.111+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8015020 con 0x7ff9d8102490 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.112+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d81a0670 con 0x7ff9d8102490 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.112+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c8011ce0 con 0x7ff9d8102490 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.112+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8010d60 con 0x7ff9d8102490 2024-06-02T15:35:08.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.112+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a0c30 con 0x7ff9d8102490 2024-06-02T15:35:08.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.113+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d806a260 con 0x7ff9d8102490 2024-06-02T15:35:08.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.115+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff9c8018020 con 0x7ff9d8102490 2024-06-02T15:35:08.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.115+0000 7ff9dc8ba700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 0x7ff9c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:08.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.115+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7ff9c80e1480 con 0x7ff9d8102490 2024-06-02T15:35:08.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.116+0000 7ff9cffff700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 0x7ff9c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:08.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.117+0000 7ff9cffff700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 0x7ff9c40b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0006010 tx=0x7ff9c0003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:35:08.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.119+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9d806a260 con 0x7ff9d8102490 2024-06-02T15:35:08.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:08.330+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff9d806bb10 con 0x7ff9c40b2880 2024-06-02T15:35:08.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:08 smithi120 ceph-mon[36451]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:08.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:08 smithi120 ceph-mon[36451]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:08.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:08.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:08.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:08.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:08 smithi135 ceph-mon[36839]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:08.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:08 smithi135 ceph-mon[36839]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:08.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:08.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:08.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:08.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:08 smithi090 ceph-mon[31595]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:08 smithi090 ceph-mon[31595]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:11.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:10 smithi120 ceph-mon[36451]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:11.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:10 smithi135 ceph-mon[36839]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:11.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:10 smithi090 ceph-mon[31595]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/1271811270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]': finished 2024-06-02T15:35:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: osdmap e51: 9 total, 8 up, 9 in 2024-06-02T15:35:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/1008382655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:12.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/1271811270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]': finished 2024-06-02T15:35:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: osdmap e51: 9 total, 8 up, 9 in 2024-06-02T15:35:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/1008382655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/1271811270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]: dispatch 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d2ed37-a237-4c17-a69e-ce75f99dc563"}]': finished 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: osdmap e51: 9 total, 8 up, 9 in 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/1008382655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:13 smithi090 ceph-mon[31595]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:35:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:13 smithi120 ceph-mon[36451]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:35:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:13 smithi135 ceph-mon[36839]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:35:15.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:15 smithi135 ceph-mon[36839]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-02T15:35:15.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:15 smithi090 ceph-mon[31595]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-02T15:35:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:15 smithi120 ceph-mon[36451]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-02T15:35:17.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:17 smithi135 ceph-mon[36839]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:17.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:17 smithi090 ceph-mon[31595]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:17 smithi120 ceph-mon[36451]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:19.292 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:19 smithi135 ceph-mon[36839]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:19.292 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T15:35:19.292 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:19.360 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:19 smithi120 ceph-mon[36451]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:19.360 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T15:35:19.360 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:19 smithi090 ceph-mon[31595]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T15:35:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:20.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:20 smithi135 ceph-mon[36839]: Deploying daemon osd.8 on smithi135 2024-06-02T15:35:20.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:20 smithi090 ceph-mon[31595]: Deploying daemon osd.8 on smithi135 2024-06-02T15:35:20.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:20 smithi120 ceph-mon[36451]: Deploying daemon osd.8 on smithi135 2024-06-02T15:35:21.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:21 smithi090 ceph-mon[31595]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:21.432 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:21 smithi135 ceph-mon[36839]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:21.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:21 smithi120 ceph-mon[36451]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:23 smithi090 ceph-mon[31595]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:23.519 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:23 smithi135 ceph-mon[36839]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:23 smithi120 ceph-mon[36451]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:25.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:25 smithi090 ceph-mon[31595]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:25.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:25.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:25 smithi120 ceph-mon[36451]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:25.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:25.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:25.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:25 smithi135 ceph-mon[36839]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:25.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:25.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:27 smithi090 ceph-mon[31595]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:27.431 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:27 smithi135 ceph-mon[36839]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:27.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:27 smithi120 ceph-mon[36451]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:27.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.814+0000 7ff9dc8ba700 1 -- 172.21.15.135:0/1478582154 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9d806bb10 con 0x7ff9c40b2880 2024-06-02T15:35:27.813 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 8 on host 'smithi135' 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 msgr2=0x7ff9c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 0x7ff9c40b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0006010 tx=0x7ff9c0003860 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 msgr2=0x7ff9d8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800b530 tx=0x7ff9c800b8f0 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 shutdown_connections 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9c40b2880 0x7ff9c40b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9d8104030 0x7ff9d819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9d8102490 0x7ff9d8198720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 --2- 172.21.15.135:0/1478582154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9d8103690 0x7ff9d8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 >> 172.21.15.135:0/1478582154 conn(0x7ff9d80fda30 msgr2=0x7ff9d8106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:35:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 shutdown_connections 2024-06-02T15:35:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:35:27.817+0000 7ff9dd8bc700 1 -- 172.21.15.135:0/1478582154 wait complete. 2024-06-02T15:35:28.534 DEBUG:teuthology.orchestra.run.smithi135:osd.8> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.8.service 2024-06-02T15:35:28.537 INFO:tasks.cephadm:Deploying osd.9 on smithi090 with /dev/vg_nvme/lv_1... 2024-06-02T15:35:28.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T15:35:28.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:28.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:28.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:28 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:28.981 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:35:29.022 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:28 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:28 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:30.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:29 smithi135 ceph-mon[36839]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T15:35:30.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:29 smithi135 ceph-mon[36839]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:29 smithi135 ceph-mon[36839]: osdmap e52: 9 total, 8 up, 9 in 2024-06-02T15:35:30.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:29 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:30.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:29 smithi135 ceph-mon[36839]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:29 smithi090 ceph-mon[31595]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T15:35:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:29 smithi090 ceph-mon[31595]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:29 smithi090 ceph-mon[31595]: osdmap e52: 9 total, 8 up, 9 in 2024-06-02T15:35:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:29 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:29 smithi090 ceph-mon[31595]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:29 smithi120 ceph-mon[36451]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T15:35:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:29 smithi120 ceph-mon[36451]: from='osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:29 smithi120 ceph-mon[36451]: osdmap e52: 9 total, 8 up, 9 in 2024-06-02T15:35:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:29 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:29 smithi120 ceph-mon[36451]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:35:30.817 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:35:30.532+0000 7f46f6224700 -1 osd.8 0 waiting for initial osdmap 2024-06-02T15:35:30.817 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:35:30.543+0000 7f46ee00f700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:35:31.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: osdmap e53: 9 total, 8 up, 9 in 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: from='osd.8 ' entity='osd.8' 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] boot 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: osdmap e54: 9 total, 9 up, 9 in 2024-06-02T15:35:31.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:31.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: osdmap e53: 9 total, 8 up, 9 in 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: from='osd.8 ' entity='osd.8' 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] boot 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: osdmap e54: 9 total, 9 up, 9 in 2024-06-02T15:35:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: osdmap e53: 9 total, 8 up, 9 in 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:31.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: from='osd.8 ' entity='osd.8' 2024-06-02T15:35:31.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: osd.8 [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] boot 2024-06-02T15:35:31.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: osdmap e54: 9 total, 9 up, 9 in 2024-06-02T15:35:31.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T15:35:32.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:31 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:35:32.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:31 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:35:32.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:31 smithi090 ceph-mon[31595]: osdmap e55: 9 total, 9 up, 9 in 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0400858 s, 262 MB/s 2024-06-02T15:35:32.077 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-06-02T15:35:32.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:31 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:35:32.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:31 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:35:32.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:31 smithi120 ceph-mon[36451]: osdmap e55: 9 total, 9 up, 9 in 2024-06-02T15:35:32.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:31 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:35:32.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:31 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:35:32.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:31 smithi135 ceph-mon[36839]: osdmap e55: 9 total, 9 up, 9 in 2024-06-02T15:35:32.913 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: osdmap e56: 9 total, 9 up, 9 in 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi135 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: osdmap e56: 9 total, 9 up, 9 in 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi135 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: osdmap e56: 9 total, 9 up, 9 in 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi135 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi135 to 5696M 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:33.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:33.349 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:35:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:34 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:34 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:34 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:34 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:34 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:34 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:34.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:34 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:34 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:35:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:34 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:35.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:35 smithi090 ceph-mon[31595]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:35.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 -- 172.21.15.90:0/4232034627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 msgr2=0x7f862010b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:35.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 --2- 172.21.15.90:0/4232034627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f862010b620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f861800b0a0 tx=0x7f861800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 -- 172.21.15.90:0/4232034627 shutdown_connections 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 --2- 172.21.15.90:0/4232034627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f862010b620 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 --2- 172.21.15.90:0/4232034627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f8620106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 --2- 172.21.15.90:0/4232034627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 0x7f8620108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 -- 172.21.15.90:0/4232034627 >> 172.21.15.90:0/4232034627 conn(0x7f86200fd960 msgr2=0x7f86200ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.533+0000 7f861fe00700 1 -- 172.21.15.90:0/4232034627 shutdown_connections 2024-06-02T15:35:35.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.534+0000 7f861fe00700 1 -- 172.21.15.90:0/4232034627 wait complete. 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 Processor -- start 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 -- start start 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f862019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 0x7f862019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f86201a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:35.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f862010efd0 con 0x7f86201091b0 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f862010ee50 con 0x7f8620106880 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.535+0000 7f861fe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f862010f150 con 0x7f8620103f50 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f86201a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f862019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861ea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f862019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60706/0 (socket says 172.21.15.90:60706) 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861ea00700 1 -- 172.21.15.90:0/1734325835 learned_addr learned my addr 172.21.15.90:0/1734325835 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:35:35.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861e000700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 0x7f862019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 -- 172.21.15.90:0/1734325835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 msgr2=0x7f862019b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f862019b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 -- 172.21.15.90:0/1734325835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 msgr2=0x7f862019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 0x7f862019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 -- 172.21.15.90:0/1734325835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86201a3410 con 0x7f86201091b0 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.536+0000 7f861f400700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f86201a2d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f861800b0a0 tx=0x7f8618004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.537+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618021070 con 0x7f86201091b0 2024-06-02T15:35:35.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.537+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8618004e50 con 0x7f86201091b0 2024-06-02T15:35:35.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.537+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86201a36a0 con 0x7f86201091b0 2024-06-02T15:35:35.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.537+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861801de60 con 0x7f86201091b0 2024-06-02T15:35:35.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.537+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201a3b90 con 0x7f86201091b0 2024-06-02T15:35:35.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.539+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862006a230 con 0x7f86201091b0 2024-06-02T15:35:35.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.541+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8618024070 con 0x7f86201091b0 2024-06-02T15:35:35.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.542+0000 7f8624a00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 0x7f86040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:35:35.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.542+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f86180e2b00 con 0x7f86201091b0 2024-06-02T15:35:35.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.543+0000 7f861ea00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 0x7f86040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:35:35.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.543+0000 7f861ea00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 0x7f86040b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8610009d20 tx=0x7f8610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:35:35.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.547+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86180ab1c0 con 0x7f86201091b0 2024-06-02T15:35:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:35 smithi120 ceph-mon[36451]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:35 smithi135 ceph-mon[36839]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:35:35.789+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8620063980 con 0x7f86040b2900 2024-06-02T15:35:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:35:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:35:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:37 smithi090 ceph-mon[31595]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-02T15:35:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:37 smithi090 ceph-mon[31595]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:37 smithi120 ceph-mon[36451]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-02T15:35:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:37 smithi120 ceph-mon[36451]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:37.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:37 smithi135 ceph-mon[36839]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 495 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-06-02T15:35:37.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:37 smithi135 ceph-mon[36839]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:35:39.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:39 smithi090 ceph-mon[31595]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-02T15:35:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:39 smithi120 ceph-mon[36451]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-02T15:35:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:39 smithi135 ceph-mon[36839]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-06-02T15:35:40.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2264835171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:40 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:40 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]': finished 2024-06-02T15:35:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:40 smithi090 ceph-mon[31595]: osdmap e57: 10 total, 9 up, 10 in 2024-06-02T15:35:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:35:40.468 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2264835171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.468 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:40 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.468 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:40 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]': finished 2024-06-02T15:35:40.469 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:40 smithi135 ceph-mon[36839]: osdmap e57: 10 total, 9 up, 10 in 2024-06-02T15:35:40.469 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:35:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2264835171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:40 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]: dispatch 2024-06-02T15:35:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:40 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0d14146-6d2a-4b23-bbd2-318bf2dad564"}]': finished 2024-06-02T15:35:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:40 smithi120 ceph-mon[36451]: osdmap e57: 10 total, 9 up, 10 in 2024-06-02T15:35:40.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:35:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:41 smithi090 ceph-mon[31595]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:41 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4125622073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:41.481 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:41 smithi135 ceph-mon[36839]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:41.481 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:41 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4125622073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:41 smithi120 ceph-mon[36451]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-02T15:35:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:41 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4125622073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:35:43.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:43 smithi090 ceph-mon[31595]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-02T15:35:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:43 smithi120 ceph-mon[36451]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-02T15:35:43.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:43 smithi135 ceph-mon[36839]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-02T15:35:45.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:45 smithi090 ceph-mon[31595]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:45 smithi120 ceph-mon[36451]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:45.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:45 smithi135 ceph-mon[36839]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T15:35:47.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:47 smithi090 ceph-mon[31595]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:47.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:47 smithi120 ceph-mon[36451]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:47.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:47 smithi135 ceph-mon[36839]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:49.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:49 smithi090 ceph-mon[31595]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:49 smithi120 ceph-mon[36451]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:49.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:49 smithi135 ceph-mon[36839]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:50.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T15:35:50.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T15:35:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T15:35:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:35:51.422 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:51 smithi090 ceph-mon[31595]: Deploying daemon osd.9 on smithi090 2024-06-02T15:35:51.422 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:51 smithi090 ceph-mon[31595]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:51 smithi120 ceph-mon[36451]: Deploying daemon osd.9 on smithi090 2024-06-02T15:35:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:51 smithi120 ceph-mon[36451]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:51.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:51 smithi135 ceph-mon[36839]: Deploying daemon osd.9 on smithi090 2024-06-02T15:35:51.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:51 smithi135 ceph-mon[36839]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:53.555 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:53 smithi090 ceph-mon[31595]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:53 smithi120 ceph-mon[36451]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:53 smithi135 ceph-mon[36839]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:55.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:55 smithi090 ceph-mon[31595]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:55 smithi120 ceph-mon[36451]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:55 smithi135 ceph-mon[36839]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:57.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:57 smithi090 ceph-mon[31595]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:57 smithi120 ceph-mon[36451]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:57 smithi135 ceph-mon[36839]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:58 smithi090 ceph-mon[31595]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:35:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:59.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:58 smithi120 ceph-mon[36451]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:59.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:35:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:35:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:58 smithi135 ceph-mon[36839]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:35:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:35:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:35:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:01.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:00 smithi090 ceph-mon[31595]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:01.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:00 smithi120 ceph-mon[36451]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:01.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:00 smithi135 ceph-mon[36839]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:02.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.927+0000 7f8624a00700 1 -- 172.21.15.90:0/1734325835 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8620063980 con 0x7f86040b2900 2024-06-02T15:36:02.927 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 9 on host 'smithi090' 2024-06-02T15:36:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 msgr2=0x7f86040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 0x7f86040b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8610009d20 tx=0x7f8610009260 comp rx=0 tx=0).stop 2024-06-02T15:36:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 msgr2=0x7f86201a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f86201a2d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f861800b0a0 tx=0x7f8618004c60 comp rx=0 tx=0).stop 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 shutdown_connections 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f86040b2900 0x7f86040b4dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86201091b0 0x7f86201a2d70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8620103f50 0x7f862019b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 --2- 172.21.15.90:0/1734325835 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8620106880 0x7f862019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 >> 172.21.15.90:0/1734325835 conn(0x7f86200fd960 msgr2=0x7f8620103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 shutdown_connections 2024-06-02T15:36:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:02.931+0000 7f861fe00700 1 -- 172.21.15.90:0/1734325835 wait complete. 2024-06-02T15:36:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:02 smithi120 ceph-mon[36451]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:02 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:02 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:02 smithi120 ceph-mon[36451]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:02 smithi120 ceph-mon[36451]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:02 smithi135 ceph-mon[36839]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:02 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:02 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:02 smithi135 ceph-mon[36839]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:02 smithi135 ceph-mon[36839]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:02 smithi090 ceph-mon[31595]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:03.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:02 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:02 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:02 smithi090 ceph-mon[31595]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:02 smithi090 ceph-mon[31595]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T15:36:03.775 DEBUG:teuthology.orchestra.run.smithi090:osd.9> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.9.service 2024-06-02T15:36:03.782 INFO:tasks.cephadm:Deploying osd.10 on smithi120 with /dev/vg_nvme/lv_1... 2024-06-02T15:36:03.783 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T15:36:04.028 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:36:04.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T15:36:04.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: osdmap e58: 10 total, 9 up, 10 in 2024-06-02T15:36:04.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:04.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: osdmap e58: 10 total, 9 up, 10 in 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: osdmap e58: 10 total, 9 up, 10 in 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-06-02T15:36:04.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:04.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:36:04.597+0000 7f3538600700 -1 osd.9 0 waiting for initial osdmap 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: osdmap e59: 10 total, 9 up, 10 in 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:04 smithi120 ceph-mon[36451]: from='osd.9 ' entity='osd.9' 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: osdmap e59: 10 total, 9 up, 10 in 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:04 smithi135 ceph-mon[36839]: from='osd.9 ' entity='osd.9' 2024-06-02T15:36:05.123 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:36:04.628+0000 7f352e400700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: osdmap e59: 10 total, 9 up, 10 in 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:04 smithi090 ceph-mon[31595]: from='osd.9 ' entity='osd.9' 2024-06-02T15:36:05.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:36:05.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:36:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] boot 2024-06-02T15:36:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: osdmap e60: 10 total, 10 up, 10 in 2024-06-02T15:36:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:05 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] boot 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: osdmap e60: 10 total, 10 up, 10 in 2024-06-02T15:36:06.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:05 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: osd.9 [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] boot 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: osdmap e60: 10 total, 10 up, 10 in 2024-06-02T15:36:06.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:05 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr: stderr: 10+0 records in 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:10+0 records out 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127921 s, 820 MB/s 2024-06-02T15:36:06.303 INFO:teuthology.orchestra.run.smithi120.stderr:--> Zapping successful for: 2024-06-02T15:36:06.978 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:06 smithi120 ceph-mon[36451]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:07.006 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi120:vg_nvme/lv_1 2024-06-02T15:36:07.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:06 smithi090 ceph-mon[31595]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:07.250 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:36:07.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:06 smithi135 ceph-mon[36839]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-06-02T15:36:08.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:07 smithi090 ceph-mon[31595]: osdmap e61: 10 total, 10 up, 10 in 2024-06-02T15:36:08.161 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:07 smithi120 ceph-mon[36451]: osdmap e61: 10 total, 10 up, 10 in 2024-06-02T15:36:08.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:07 smithi135 ceph-mon[36839]: osdmap e61: 10 total, 10 up, 10 in 2024-06-02T15:36:08.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1077192068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 msgr2=0x7fdf181028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:08.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1077192068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf181028d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf080099c0 tx=0x7fdf08009cd0 comp rx=0 tx=0).stop 2024-06-02T15:36:08.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1077192068 shutdown_connections 2024-06-02T15:36:08.999 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1077192068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf18104050 0x7fdf1810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1077192068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf181028d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1077192068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf181036b0 0x7fdf18103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1077192068 >> 172.21.15.120:0/1077192068 conn(0x7fdf180fda70 msgr2=0x7fdf180ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1077192068 shutdown_connections 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.001+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1077192068 wait complete. 2024-06-02T15:36:09.000 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 Processor -- start 2024-06-02T15:36:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 -- start start 2024-06-02T15:36:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf181036b0 0x7fdf18198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf18104050 0x7fdf1819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:09.002 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810dfd0 con 0x7fdf181036b0 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810de50 con 0x7fdf181024b0 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1cff7700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810e150 con 0x7fdf18104050 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:34838/0 (socket says 172.21.15.120:34838) 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 -- 172.21.15.120:0/1780366016 learned_addr learned my addr 172.21.15.120:0/1780366016 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 -- 172.21.15.120:0/1780366016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf18104050 msgr2=0x7fdf1819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:09.003 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf18104050 0x7fdf1819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 -- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf181036b0 msgr2=0x7fdf18198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf181036b0 0x7fdf18198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.002+0000 7fdf1659c700 1 -- 172.21.15.120:0/1780366016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf181a03a0 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf1659c700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009f50 tx=0x7fdf08003750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08027070 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf181a0630 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf08022470 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800f6c0 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.003+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf181a0b10 con 0x7fdf181024b0 2024-06-02T15:36:09.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.004+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1806a230 con 0x7fdf181024b0 2024-06-02T15:36:09.005 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.005+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdf0802a070 con 0x7fdf181024b0 2024-06-02T15:36:09.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.006+0000 7fdf0ffff700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 0x7fdf040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:09.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.006+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7fdf080e2ab0 con 0x7fdf181024b0 2024-06-02T15:36:09.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.006+0000 7fdf0dd9b700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 0x7fdf040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:09.006 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.007+0000 7fdf0dd9b700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 0x7fdf040b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097b0 tx=0x7fdf00006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:09.007 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.009+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf08013070 con 0x7fdf181024b0 2024-06-02T15:36:09.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:08 smithi120 ceph-mon[36451]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:09.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:08 smithi090 ceph-mon[31595]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:09.220 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:09.222+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdf18063980 con 0x7fdf040b2880 2024-06-02T15:36:09.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:08 smithi135 ceph-mon[36839]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi090 2024-06-02T15:36:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 4272M 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi090 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 4272M 2024-06-02T15:36:10.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi090 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 4272M 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi120:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:10.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:11.221 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:11 smithi120 ceph-mon[36451]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:11.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:11 smithi090 ceph-mon[31595]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:11.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:11 smithi135 ceph-mon[36839]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/2444011730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]': finished 2024-06-02T15:36:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: osdmap e62: 11 total, 10 up, 11 in 2024-06-02T15:36:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.120:0/72222649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/2444011730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]': finished 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: osdmap e62: 11 total, 10 up, 11 in 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:13.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:13 smithi120 ceph-mon[36451]: from='client.? 172.21.15.120:0/72222649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/2444011730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]: dispatch 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a96be6b1-b9f1-4e56-8394-092362a013ca"}]': finished 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: osdmap e62: 11 total, 10 up, 11 in 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:13.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:13 smithi135 ceph-mon[36839]: from='client.? 172.21.15.120:0/72222649' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:15.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:15 smithi090 ceph-mon[31595]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:15 smithi120 ceph-mon[36451]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:15.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:15 smithi135 ceph-mon[36839]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:17.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:17 smithi090 ceph-mon[31595]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:17.441 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:17 smithi120 ceph-mon[36451]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:17 smithi135 ceph-mon[36839]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:19.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:19 smithi120 ceph-mon[36451]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:19.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:19 smithi135 ceph-mon[36839]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:19 smithi090 ceph-mon[31595]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:20.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T15:36:20.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:20.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T15:36:20.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T15:36:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:21 smithi120 ceph-mon[36451]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:21 smithi120 ceph-mon[36451]: Deploying daemon osd.10 on smithi120 2024-06-02T15:36:21.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:21 smithi135 ceph-mon[36839]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:21.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:21 smithi135 ceph-mon[36839]: Deploying daemon osd.10 on smithi120 2024-06-02T15:36:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:21 smithi090 ceph-mon[31595]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:21 smithi090 ceph-mon[31595]: Deploying daemon osd.10 on smithi120 2024-06-02T15:36:23.468 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:23 smithi120 ceph-mon[36451]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:23.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:23 smithi135 ceph-mon[36839]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:23 smithi090 ceph-mon[31595]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:25.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:25 smithi120 ceph-mon[36451]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:25.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:25 smithi135 ceph-mon[36839]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:25 smithi090 ceph-mon[31595]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:27 smithi120 ceph-mon[36451]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:27.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:27 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:27 smithi135 ceph-mon[36839]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:27 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:27 smithi090 ceph-mon[31595]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:27 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:29 smithi120 ceph-mon[36451]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:29 smithi135 ceph-mon[36839]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:29.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:29 smithi090 ceph-mon[31595]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:30.107 INFO:teuthology.orchestra.run.smithi120.stdout:Created osd(s) 10 on host 'smithi120' 2024-06-02T15:36:30.107 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.108+0000 7fdf0ffff700 1 -- 172.21.15.120:0/1780366016 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdf18063980 con 0x7fdf040b2880 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 msgr2=0x7fdf040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 0x7fdf040b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097b0 tx=0x7fdf00006d20 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 msgr2=0x7fdf18198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009f50 tx=0x7fdf08003750 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 shutdown_connections 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf040b2880 0x7fdf040b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf18104050 0x7fdf1819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf181024b0 0x7fdf18198790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 --2- 172.21.15.120:0/1780366016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf181036b0 0x7fdf18198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 >> 172.21.15.120:0/1780366016 conn(0x7fdf180fda70 msgr2=0x7fdf18106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 shutdown_connections 2024-06-02T15:36:30.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-06-02T15:36:30.111+0000 7fdf1cff7700 1 -- 172.21.15.120:0/1780366016 wait complete. 2024-06-02T15:36:30.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:30.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:30.949 DEBUG:teuthology.orchestra.run.smithi120:osd.10> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.10.service 2024-06-02T15:36:30.951 INFO:tasks.cephadm:Deploying osd.11 on smithi135 with /dev/vg_nvme/lv_1... 2024-06-02T15:36:30.952 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T15:36:31.199 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:36:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T15:36:31.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:31.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: osdmap e63: 11 total, 10 up, 11 in 2024-06-02T15:36:31.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:31.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:31 smithi120 ceph-mon[36451]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T15:36:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: osdmap e63: 11 total, 10 up, 11 in 2024-06-02T15:36:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:31 smithi090 ceph-mon[31595]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:31.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:31.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T15:36:31.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: from='osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:31.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: osdmap e63: 11 total, 10 up, 11 in 2024-06-02T15:36:31.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:31.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:31 smithi135 ceph-mon[36839]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]: dispatch 2024-06-02T15:36:32.298 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:36:31.998+0000 7f5f15d42700 -1 osd.10 0 waiting for initial osdmap 2024-06-02T15:36:32.298 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:36:32.012+0000 7f5f0db2d700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:36:32.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: osdmap e64: 11 total, 10 up, 11 in 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: from='osd.10 ' entity='osd.10' 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] boot 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: osdmap e65: 11 total, 11 up, 11 in 2024-06-02T15:36:32.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:36:32.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: osdmap e64: 11 total, 10 up, 11 in 2024-06-02T15:36:32.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: from='osd.10 ' entity='osd.10' 2024-06-02T15:36:32.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] boot 2024-06-02T15:36:32.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: osdmap e65: 11 total, 11 up, 11 in 2024-06-02T15:36:32.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi120", "root=default"]}]': finished 2024-06-02T15:36:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: osdmap e64: 11 total, 10 up, 11 in 2024-06-02T15:36:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: from='osd.10 ' entity='osd.10' 2024-06-02T15:36:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: osd.10 [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] boot 2024-06-02T15:36:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: osdmap e65: 11 total, 11 up, 11 in 2024-06-02T15:36:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113532 s, 924 MB/s 2024-06-02T15:36:33.340 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-06-02T15:36:33.473 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:33 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:36:33.473 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:33 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:36:33.473 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:33 smithi120 ceph-mon[36451]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:33.473 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:33 smithi120 ceph-mon[36451]: osdmap e66: 11 total, 11 up, 11 in 2024-06-02T15:36:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:33 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:36:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:33 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:36:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:33 smithi090 ceph-mon[31595]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:33 smithi090 ceph-mon[31595]: osdmap e66: 11 total, 11 up, 11 in 2024-06-02T15:36:33.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:33 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:36:33.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:33 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:36:33.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:33 smithi135 ceph-mon[36839]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-06-02T15:36:33.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:33 smithi135 ceph-mon[36839]: osdmap e66: 11 total, 11 up, 11 in 2024-06-02T15:36:34.018 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-06-02T15:36:34.254 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:36:35.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:35.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi120 2024-06-02T15:36:35.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.414 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: osdmap e67: 11 total, 11 up, 11 in 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-02T15:36:35.416 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.416 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:35.416 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:35.416 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi120 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: osdmap e67: 11 total, 11 up, 11 in 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi120 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: osdmap e67: 11 total, 11 up, 11 in 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:36:35.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 4272M 2024-06-02T15:36:35.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:35.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:36:35.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:35.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/3016475716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f54104060 msgr2=0x7f4f5410a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/3016475716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f54104060 0x7f4f5410a910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b0a0 tx=0x7f4f4c00b3b0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/3016475716 shutdown_connections 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/3016475716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f54104060 0x7f4f5410a910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/3016475716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f541024c0 0x7f4f541028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/3016475716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f54103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/3016475716 >> 172.21.15.135:0/3016475716 conn(0x7f4f540fdaa0 msgr2=0x7f4f540ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/3016475716 shutdown_connections 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/3016475716 wait complete. 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 Processor -- start 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- start start 2024-06-02T15:36:35.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f5419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54104060 0x7f4f541a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410dfe0 con 0x7f4f54104060 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410de60 con 0x7f4f541036c0 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.931+0000 7f4f5b8b9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410e160 con 0x7f4f541024c0 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54874/0 (socket says 172.21.15.135:54874) 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 -- 172.21.15.135:0/1510108434 learned_addr learned my addr 172.21.15.135:0/1510108434 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f58e54700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f5419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 -- 172.21.15.135:0/1510108434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 msgr2=0x7f4f5419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:35.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f5419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 -- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54104060 msgr2=0x7f4f541a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54104060 0x7f4f541a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 -- 172.21.15.135:0/1510108434 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541a1350 con 0x7f4f541024c0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f58e54700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f5419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f59655700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000b500 tx=0x7f4f5000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50015020 con 0x7f4f541024c0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f50011ce0 con 0x7f4f541024c0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a1640 con 0x7f4f541024c0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50010d60 con 0x7f4f541024c0 2024-06-02T15:36:35.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.932+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a1b80 con 0x7f4f541024c0 2024-06-02T15:36:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.933+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5406a230 con 0x7f4f541024c0 2024-06-02T15:36:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.935+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4f50018020 con 0x7f4f541024c0 2024-06-02T15:36:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.936+0000 7f4f4a7fc700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 0x7f4f400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.936+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f4f500e1950 con 0x7f4f541024c0 2024-06-02T15:36:35.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.937+0000 7f4f58e54700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 0x7f4f400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:35.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.937+0000 7f4f58e54700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 0x7f4f400b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005fd0 tx=0x7f4f44005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:35.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:35.940+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f5001f020 con 0x7f4f541024c0 2024-06-02T15:36:36.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:36.148+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4f54063980 con 0x7f4f400b2880 2024-06-02T15:36:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:36.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T15:36:36.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T15:36:36.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:37.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:37 smithi090 ceph-mon[31595]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-06-02T15:36:37.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:37 smithi090 ceph-mon[31595]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:37.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:37 smithi135 ceph-mon[36839]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-06-02T15:36:37.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:37 smithi135 ceph-mon[36839]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:37.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:37 smithi120 ceph-mon[36451]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-06-02T15:36:37.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:37 smithi120 ceph-mon[36451]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:36:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:36:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/1787519110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]': finished 2024-06-02T15:36:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: osdmap e68: 12 total, 11 up, 12 in 2024-06-02T15:36:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:36:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/1787519110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]': finished 2024-06-02T15:36:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: osdmap e68: 12 total, 11 up, 12 in 2024-06-02T15:36:39.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/1787519110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]: dispatch 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fc0220b-d39c-4245-8784-8334b9d06f67"}]': finished 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: osdmap e68: 12 total, 11 up, 12 in 2024-06-02T15:36:39.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:40.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.135:0/3496659806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:40.718 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.135:0/3496659806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:40.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.135:0/3496659806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T15:36:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:41 smithi090 ceph-mon[31595]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:36:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:41 smithi120 ceph-mon[36451]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:36:41.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:41 smithi135 ceph-mon[36839]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-06-02T15:36:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:43 smithi090 ceph-mon[31595]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:36:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:43 smithi120 ceph-mon[36451]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:36:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:43 smithi135 ceph-mon[36839]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-02T15:36:45.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:45 smithi135 ceph-mon[36839]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:36:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:45 smithi090 ceph-mon[31595]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:36:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:45 smithi120 ceph-mon[36451]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-06-02T15:36:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:46 smithi120 ceph-mon[36451]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:46.870 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T15:36:46.870 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:46.871 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:46 smithi135 ceph-mon[36839]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:46.871 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T15:36:46.871 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:46 smithi090 ceph-mon[31595]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T15:36:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:36:47.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:47 smithi120 ceph-mon[36451]: Deploying daemon osd.11 on smithi135 2024-06-02T15:36:47.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:47 smithi135 ceph-mon[36839]: Deploying daemon osd.11 on smithi135 2024-06-02T15:36:47.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:47 smithi090 ceph-mon[31595]: Deploying daemon osd.11 on smithi135 2024-06-02T15:36:48.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:48 smithi135 ceph-mon[36839]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:48 smithi120 ceph-mon[36451]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:48.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:48 smithi090 ceph-mon[31595]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:51.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:50 smithi120 ceph-mon[36451]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:50 smithi135 ceph-mon[36839]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:51.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:50 smithi090 ceph-mon[31595]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:53 smithi120 ceph-mon[36451]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:53.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:53 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:53 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:53 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:53 smithi135 ceph-mon[36839]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:53 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:53 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:53 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:53 smithi090 ceph-mon[31595]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:53 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:53 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:53.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:53 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:36:55.508 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:55 smithi135 ceph-mon[36839]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:55 smithi120 ceph-mon[36451]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:55 smithi090 ceph-mon[31595]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:56.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.197+0000 7f4f4a7fc700 1 -- 172.21.15.135:0/1510108434 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4f54063980 con 0x7f4f400b2880 2024-06-02T15:36:56.195 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 11 on host 'smithi135' 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 msgr2=0x7f4f400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 0x7f4f400b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f44005fd0 tx=0x7f4f44005f60 comp rx=0 tx=0).stop 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 msgr2=0x7f4f5406c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000b500 tx=0x7f4f5000b8c0 comp rx=0 tx=0).stop 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 shutdown_connections 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4f400b2880 0x7f4f400b4d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:56.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54104060 0x7f4f541a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:56.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f541024c0 0x7f4f5406c870 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:56.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 --2- 172.21.15.135:0/1510108434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f541036c0 0x7f4f5419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:56.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 >> 172.21.15.135:0/1510108434 conn(0x7f4f540fdaa0 msgr2=0x7f4f54106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:56.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 shutdown_connections 2024-06-02T15:36:56.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:36:56.199+0000 7f4f5b8b9700 1 -- 172.21.15.135:0/1510108434 wait complete. 2024-06-02T15:36:56.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:56 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:56 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:56 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:36:56.918 DEBUG:teuthology.orchestra.run.smithi135:osd.11> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.11.service 2024-06-02T15:36:56.920 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-02T15:36:56.920 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-02T15:36:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:36:57.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-mon[36839]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T15:36:57.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-mon[36839]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:57.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-mon[36839]: osdmap e69: 12 total, 11 up, 12 in 2024-06-02T15:36:57.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:57.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-mon[36839]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:57.817 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:36:57.559+0000 7f4e27cee700 -1 osd.11 0 waiting for initial osdmap 2024-06-02T15:36:57.818 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:36:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:36:57.570+0000 7f4e23ae1700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T15:36:57.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:57 smithi090 ceph-mon[31595]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T15:36:57.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:57 smithi090 ceph-mon[31595]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:57.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:57 smithi090 ceph-mon[31595]: osdmap e69: 12 total, 11 up, 12 in 2024-06-02T15:36:57.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:57.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:57 smithi090 ceph-mon[31595]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:58.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:57 smithi120 ceph-mon[36451]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T15:36:58.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:57 smithi120 ceph-mon[36451]: from='osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:57 smithi120 ceph-mon[36451]: osdmap e69: 12 total, 11 up, 12 in 2024-06-02T15:36:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:58.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:57 smithi120 ceph-mon[36451]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-06-02T15:36:58.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:58 smithi135 ceph-mon[36839]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:36:58.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:58 smithi135 ceph-mon[36839]: osdmap e70: 12 total, 11 up, 12 in 2024-06-02T15:36:58.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:58.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:58 smithi135 ceph-mon[36839]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:58.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:58.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:58 smithi090 ceph-mon[31595]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:36:58.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:58 smithi090 ceph-mon[31595]: osdmap e70: 12 total, 11 up, 12 in 2024-06-02T15:36:58.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:58.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:58 smithi090 ceph-mon[31595]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:58.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:58 smithi120 ceph-mon[36451]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-06-02T15:36:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:58 smithi120 ceph-mon[36451]: osdmap e70: 12 total, 11 up, 12 in 2024-06-02T15:36:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:58 smithi120 ceph-mon[36451]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-06-02T15:36:59.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:59.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 -- 172.21.15.90:0/3375674816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 msgr2=0x7efe701038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:59.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 --2- 172.21.15.90:0/3375674816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe701038f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efe64009920 tx=0x7efe64009c30 comp rx=0 tx=0).stop 2024-06-02T15:36:59.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 -- 172.21.15.90:0/3375674816 shutdown_connections 2024-06-02T15:36:59.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 --2- 172.21.15.90:0/3375674816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe7010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 --2- 172.21.15.90:0/3375674816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe701038f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 --2- 172.21.15.90:0/3375674816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 0x7efe70104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.654+0000 7efe76a00700 1 -- 172.21.15.90:0/3375674816 >> 172.21.15.90:0/3375674816 conn(0x7efe700fea70 msgr2=0x7efe70100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:59.654 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:59 smithi090 ceph-mon[31595]: purged_snaps scrub starts 2024-06-02T15:36:59.654 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:59 smithi090 ceph-mon[31595]: purged_snaps scrub ok 2024-06-02T15:36:59.654 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:59 smithi090 ceph-mon[31595]: osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] boot 2024-06-02T15:36:59.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:59 smithi090 ceph-mon[31595]: osdmap e71: 12 total, 12 up, 12 in 2024-06-02T15:36:59.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:36:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:36:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.655+0000 7efe76a00700 1 -- 172.21.15.90:0/3375674816 shutdown_connections 2024-06-02T15:36:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.655+0000 7efe76a00700 1 -- 172.21.15.90:0/3375674816 wait complete. 2024-06-02T15:36:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.656+0000 7efe76a00700 1 Processor -- start 2024-06-02T15:36:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 -- start start 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 0x7efe70199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe701a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010ef60 con 0x7efe701034d0 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010ede0 con 0x7efe701046d0 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010f0e0 con 0x7efe70105070 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe75600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe74c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 0x7efe70199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe75600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42860/0 (socket says 172.21.15.90:42860) 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe75600700 1 -- 172.21.15.90:0/3607992117 learned_addr learned my addr 172.21.15.90:0/3607992117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.657+0000 7efe76000700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe701a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 -- 172.21.15.90:0/3607992117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 msgr2=0x7efe701a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe701a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 -- 172.21.15.90:0/3607992117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 msgr2=0x7efe70199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 0x7efe70199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 -- 172.21.15.90:0/3607992117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe701a12c0 con 0x7efe701034d0 2024-06-02T15:36:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe76000700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe701a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe75600700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe64011040 tx=0x7efe64005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.658+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64027070 con 0x7efe701034d0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.659+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe64005540 con 0x7efe701034d0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.659+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe70075770 con 0x7efe701034d0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.659+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64004a50 con 0x7efe701034d0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.659+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe70075c50 con 0x7efe701034d0 2024-06-02T15:36:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.660+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7006a230 con 0x7efe701034d0 2024-06-02T15:36:59.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.662+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efe6402a070 con 0x7efe701034d0 2024-06-02T15:36:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.664+0000 7efe6e000700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:36:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.665+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7efe640e29c0 con 0x7efe701034d0 2024-06-02T15:36:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.665+0000 7efe74c00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:36:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.666+0000 7efe74c00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 0x7efe580b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009750 tx=0x7efe5c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:36:59.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.669+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe64013030 con 0x7efe701034d0 2024-06-02T15:36:59.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.905+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efe70077030 con 0x7efe701034d0 2024-06-02T15:36:59.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.906+0000 7efe6e000700 1 -- 172.21.15.90:0/3607992117 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7efe640aab70 con 0x7efe701034d0 2024-06-02T15:36:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 msgr2=0x7efe580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 0x7efe580b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009750 tx=0x7efe5c012040 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 msgr2=0x7efe70199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe64011040 tx=0x7efe64005270 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 shutdown_connections 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe580b2880 0x7efe580b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe70105070 0x7efe701a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe701034d0 0x7efe70199660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 --2- 172.21.15.90:0/3607992117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe701046d0 0x7efe70199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 >> 172.21.15.90:0/3607992117 conn(0x7efe700fea70 msgr2=0x7efe70107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:36:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 shutdown_connections 2024-06-02T15:36:59.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:36:59.909+0000 7efe76a00700 1 -- 172.21.15.90:0/3607992117 wait complete. 2024-06-02T15:36:59.976 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:59 smithi135 ceph-mon[36839]: purged_snaps scrub starts 2024-06-02T15:36:59.976 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:59 smithi135 ceph-mon[36839]: purged_snaps scrub ok 2024-06-02T15:36:59.976 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:59 smithi135 ceph-mon[36839]: osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] boot 2024-06-02T15:36:59.976 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:59 smithi135 ceph-mon[36839]: osdmap e71: 12 total, 12 up, 12 in 2024-06-02T15:36:59.976 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:36:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:37:00.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:59 smithi120 ceph-mon[36451]: purged_snaps scrub starts 2024-06-02T15:37:00.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:59 smithi120 ceph-mon[36451]: purged_snaps scrub ok 2024-06-02T15:37:00.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:59 smithi120 ceph-mon[36451]: osd.11 [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] boot 2024-06-02T15:37:00.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:59 smithi120 ceph-mon[36451]: osdmap e71: 12 total, 12 up, 12 in 2024-06-02T15:37:00.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:36:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T15:37:00.725 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1717342618,"num_in_osds":12,"osd_in_since":1717342598,"num_remapped_pgs":0} 2024-06-02T15:37:00.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: osdmap e72: 12 total, 12 up, 12 in 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3607992117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: Detected new or changed devices on smithi135 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: osdmap e72: 12 total, 12 up, 12 in 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3607992117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: Detected new or changed devices on smithi135 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:01.059 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: osdmap e72: 12 total, 12 up, 12 in 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3607992117' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: Detected new or changed devices on smithi135 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi135 to 4272M 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:01.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:02.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:02 smithi090 ceph-mon[31595]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:03.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:02 smithi120 ceph-mon[36451]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:03.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:02 smithi135 ceph-mon[36839]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:03.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 -- 172.21.15.90:0/1016365798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 msgr2=0x7f7ce8112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:03.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 --2- 172.21.15.90:0/1016365798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce8112330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000b0a0 tx=0x7f7ce000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 -- 172.21.15.90:0/1016365798 shutdown_connections 2024-06-02T15:37:03.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 --2- 172.21.15.90:0/1016365798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce8112330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 --2- 172.21.15.90:0/1016365798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8075770 0x7f7ce8075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 --2- 172.21.15.90:0/1016365798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8076090 0x7f7ce810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 -- 172.21.15.90:0/1016365798 >> 172.21.15.90:0/1016365798 conn(0x7f7ce807a6a0 msgr2=0x7f7ce807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:03.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 -- 172.21.15.90:0/1016365798 shutdown_connections 2024-06-02T15:37:03.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.327+0000 7f7ceca00700 1 -- 172.21.15.90:0/1016365798 wait complete. 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.328+0000 7f7ceca00700 1 Processor -- start 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 -- start start 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8075770 0x7f7ce819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 0x7f7ce819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81158e0 con 0x7f7ce810ff40 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8115760 con 0x7f7ce8075770 2024-06-02T15:37:03.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ceca00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8115a60 con 0x7f7ce8076090 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ce6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ce5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 0x7f7ce819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ce5e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8075770 0x7f7ce819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ce6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36120/0 (socket says 172.21.15.90:36120) 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.329+0000 7f7ce6800700 1 -- 172.21.15.90:0/3623508317 learned_addr learned my addr 172.21.15.90:0/3623508317 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 -- 172.21.15.90:0/3623508317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8075770 msgr2=0x7f7ce819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8075770 0x7f7ce819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 -- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 msgr2=0x7f7ce81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 -- 172.21.15.90:0/3623508317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81a0de0 con 0x7f7ce8076090 2024-06-02T15:37:03.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ce5400700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 0x7f7ce819c310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9f0 tx=0x7f7cd400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:03.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd400c9c0 con 0x7f7ce8076090 2024-06-02T15:37:03.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce81a10d0 con 0x7f7ce8076090 2024-06-02T15:37:03.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd400cb60 con 0x7f7ce8076090 2024-06-02T15:37:03.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.330+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4018680 con 0x7f7ce8076090 2024-06-02T15:37:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.331+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81a8950 con 0x7f7ce8076090 2024-06-02T15:37:03.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.332+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce806a230 con 0x7f7ce8076090 2024-06-02T15:37:03.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.335+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7cd4010430 con 0x7f7ce8076090 2024-06-02T15:37:03.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.336+0000 7f7cdea00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 0x7f7ccc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.336+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f7cd40e2270 con 0x7f7ce8076090 2024-06-02T15:37:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.337+0000 7f7ce5e00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 0x7f7ccc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:03.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.337+0000 7f7ce5e00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 0x7f7ccc0b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009c60 tx=0x7f7cd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:03.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.337+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cd40aa390 con 0x7f7ce8076090 2024-06-02T15:37:03.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.570+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ce802e560 con 0x7f7ce8076090 2024-06-02T15:37:03.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.571+0000 7f7cdea00700 1 -- 172.21.15.90:0/3623508317 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18468 (secure 0 0 0) 0x7f7cd40b2410 con 0x7f7ce8076090 2024-06-02T15:37:03.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:03.571 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":72,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:36:59.577928+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:37:03.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.574+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 msgr2=0x7f7ccc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:03.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.574+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 0x7f7ccc0b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009c60 tx=0x7f7cd8009260 comp rx=0 tx=0).stop 2024-06-02T15:37:03.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.574+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 msgr2=0x7f7ce819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:03.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.574+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 0x7f7ce819c310 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400e9f0 tx=0x7f7cd400ed00 comp rx=0 tx=0).stop 2024-06-02T15:37:03.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 shutdown_connections 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ccc0b2880 0x7f7ccc0b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ce810ff40 0x7f7ce81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ce8075770 0x7f7ce819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 --2- 172.21.15.90:0/3623508317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8076090 0x7f7ce819c310 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 >> 172.21.15.90:0/3623508317 conn(0x7f7ce807a6a0 msgr2=0x7f7ce8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 shutdown_connections 2024-06-02T15:37:03.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:03.575+0000 7f7ceca00700 1 -- 172.21.15.90:0/3623508317 wait complete. 2024-06-02T15:37:03.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:03 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3623508317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:37:04.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:03 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3623508317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:37:04.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:03 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3623508317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:37:04.402 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T15:32:41.694560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-02T15:37:04.402 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-02T15:37:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:04.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:04 smithi090 ceph-mon[31595]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:05.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:04 smithi120 ceph-mon[36451]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:05.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:04 smithi135 ceph-mon[36839]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:06 smithi090 ceph-mon[31595]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:07.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:06 smithi120 ceph-mon[36451]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:07.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:06 smithi135 ceph-mon[36839]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 -- 172.21.15.90:0/2120322656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 msgr2=0x7fca6410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 --2- 172.21.15.90:0/2120322656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6410e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b0d0 tx=0x7fca5c00b3e0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 -- 172.21.15.90:0/2120322656 shutdown_connections 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 --2- 172.21.15.90:0/2120322656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6410e9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 --2- 172.21.15.90:0/2120322656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64107bd0 0x7fca64107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 --2- 172.21.15.90:0/2120322656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64101d70 0x7fca641021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.094+0000 7fca6a400700 1 -- 172.21.15.90:0/2120322656 >> 172.21.15.90:0/2120322656 conn(0x7fca640776f0 msgr2=0x7fca64077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.095+0000 7fca6a400700 1 -- 172.21.15.90:0/2120322656 shutdown_connections 2024-06-02T15:37:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.095+0000 7fca6a400700 1 -- 172.21.15.90:0/2120322656 wait complete. 2024-06-02T15:37:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 Processor -- start 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 -- start start 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 0x7fca6419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64107bd0 0x7fca641a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca64111550 con 0x7fca64102730 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fca641113d0 con 0x7fca64101d70 2024-06-02T15:37:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.096+0000 7fca6a400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca641116d0 con 0x7fca64107bd0 2024-06-02T15:37:07.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca63e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:07.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca63e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6419c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36142/0 (socket says 172.21.15.90:36142) 2024-06-02T15:37:07.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 0x7fca6419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca63e00700 1 -- 172.21.15.90:0/2861844213 learned_addr learned my addr 172.21.15.90:0/2861844213 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69a00700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64107bd0 0x7fca641a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 -- 172.21.15.90:0/2861844213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64107bd0 msgr2=0x7fca641a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64107bd0 0x7fca641a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 -- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 msgr2=0x7fca6419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 -- 172.21.15.90:0/2861844213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a0da0 con 0x7fca64101d70 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.097+0000 7fca69000700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 0x7fca6419bd90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca5800afd0 tx=0x7fca5800edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.098+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5800cb90 con 0x7fca64101d70 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.098+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca580044e0 con 0x7fca64101d70 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.098+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58005280 con 0x7fca64101d70 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.098+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a1090 con 0x7fca64101d70 2024-06-02T15:37:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.098+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a8910 con 0x7fca64101d70 2024-06-02T15:37:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.099+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6406a230 con 0x7fca64101d70 2024-06-02T15:37:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.101+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fca58004750 con 0x7fca64101d70 2024-06-02T15:37:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.103+0000 7fca61600700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 0x7fca4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.103+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fca580e1d60 con 0x7fca64101d70 2024-06-02T15:37:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.103+0000 7fca63e00700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 0x7fca4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.104+0000 7fca63e00700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 0x7fca4c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca64077b10 tx=0x7fca50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:07.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.105+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca580a9e80 con 0x7fca64101d70 2024-06-02T15:37:07.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.334+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca641a8b40 con 0x7fca64101d70 2024-06-02T15:37:07.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.335+0000 7fca61600700 1 -- 172.21.15.90:0/2861844213 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fca580a9780 con 0x7fca64101d70 2024-06-02T15:37:07.334 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-06-02T15:37:07.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.338+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 msgr2=0x7fca4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:07.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.338+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 0x7fca4c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca64077b10 tx=0x7fca50009260 comp rx=0 tx=0).stop 2024-06-02T15:37:07.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.338+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 msgr2=0x7fca6419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:07.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.338+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 0x7fca6419bd90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca5800afd0 tx=0x7fca5800edb0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 shutdown_connections 2024-06-02T15:37:07.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fca4c0b2880 0x7fca4c0b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca64107bd0 0x7fca641a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fca64101d70 0x7fca6419bd90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 --2- 172.21.15.90:0/2861844213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca64102730 0x7fca6419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:07.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 >> 172.21.15.90:0/2861844213 conn(0x7fca640776f0 msgr2=0x7fca64100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:07.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 shutdown_connections 2024-06-02T15:37:07.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:07.339+0000 7fca6a400700 1 -- 172.21.15.90:0/2861844213 wait complete. 2024-06-02T15:37:08.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:07 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2861844213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:37:08.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:07 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2861844213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:37:08.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:07 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2861844213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.a on smithi090 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.d on smithi090 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.g on smithi090 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.j on smithi090 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.b on smithi120 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.e on smithi120 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.h on smithi120 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.k on smithi120 2024-06-02T15:37:08.215 INFO:tasks.cephadm:Adding mds.c on smithi135 2024-06-02T15:37:08.216 INFO:tasks.cephadm:Adding mds.f on smithi135 2024-06-02T15:37:08.216 INFO:tasks.cephadm:Adding mds.i on smithi135 2024-06-02T15:37:08.216 INFO:tasks.cephadm:Adding mds.l on smithi135 2024-06-02T15:37:08.216 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l' 2024-06-02T15:37:08.463 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:37:08.904 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:08 smithi135 ceph-mon[36839]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:09.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:08 smithi120 ceph-mon[36451]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:09.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:08 smithi090 ceph-mon[31595]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:10.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 -- 172.21.15.135:0/1184050102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 msgr2=0x7f89a810b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:10.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 --2- 172.21.15.135:0/1184050102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a810b710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b0a0 tx=0x7f89a000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 -- 172.21.15.135:0/1184050102 shutdown_connections 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 --2- 172.21.15.135:0/1184050102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a810b710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 --2- 172.21.15.135:0/1184050102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89a8104040 0x7f89a8106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 --2- 172.21.15.135:0/1184050102 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89a8106970 0x7f89a8108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 -- 172.21.15.135:0/1184050102 >> 172.21.15.135:0/1184050102 conn(0x7f89a80fda70 msgr2=0x7f89a80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 -- 172.21.15.135:0/1184050102 shutdown_connections 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.159+0000 7f89acfb2700 1 -- 172.21.15.135:0/1184050102 wait complete. 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 Processor -- start 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 -- start start 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89a8104040 0x7f89a8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89a8106970 0x7f89a8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810f090 con 0x7f89a8106970 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810ef10 con 0x7f89a8104040 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89acfb2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810f210 con 0x7f89a81092a0 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39558/0 (socket says 172.21.15.135:39558) 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 -- 172.21.15.135:0/83205009 learned_addr learned my addr 172.21.15.135:0/83205009 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 -- 172.21.15.135:0/83205009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89a8104040 msgr2=0x7f89a8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89a8104040 0x7f89a8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 -- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89a8106970 msgr2=0x7f89a8199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89a8106970 0x7f89a8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 -- 172.21.15.135:0/83205009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a81a13d0 con 0x7f89a81092a0 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.160+0000 7f89a6d9d700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006070 tx=0x7f89a0004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:10.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.161+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0021070 con 0x7f89a81092a0 2024-06-02T15:37:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.161+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a0007db0 con 0x7f89a81092a0 2024-06-02T15:37:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.161+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a001d450 con 0x7f89a81092a0 2024-06-02T15:37:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.161+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a81a1660 con 0x7f89a81092a0 2024-06-02T15:37:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.161+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81a1b40 con 0x7f89a81092a0 2024-06-02T15:37:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.162+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a806a230 con 0x7f89a81092a0 2024-06-02T15:37:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.163+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f89a00040b0 con 0x7f89a81092a0 2024-06-02T15:37:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.163+0000 7f89977fe700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 0x7f89900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.164+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f89a00e2fb0 con 0x7f89a81092a0 2024-06-02T15:37:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.164+0000 7f89a659c700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 0x7f89900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.165+0000 7f89a659c700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 0x7f89900b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89980097b0 tx=0x7f8998006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.168+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89a00ab200 con 0x7f89a81092a0 2024-06-02T15:37:10.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.380+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}) -- 0x7f89a8063b80 con 0x7f89900b2880 2024-06-02T15:37:10.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.394+0000 7f89977fe700 1 -- 172.21.15.135:0/83205009 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f89a8063b80 con 0x7f89900b2880 2024-06-02T15:37:10.392 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mds.all update... 2024-06-02T15:37:10.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 msgr2=0x7f89900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:10.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 0x7f89900b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89980097b0 tx=0x7f8998006d20 comp rx=0 tx=0).stop 2024-06-02T15:37:10.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 msgr2=0x7f89a81a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89a0006070 tx=0x7f89a0004ef0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 shutdown_connections 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f89900b2880 0x7f89900b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89a81092a0 0x7f89a81a0d30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89a8104040 0x7f89a8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 --2- 172.21.15.135:0/83205009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89a8106970 0x7f89a8199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.396+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 >> 172.21.15.135:0/83205009 conn(0x7f89a80fda70 msgr2=0x7f89a8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.397+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 shutdown_connections 2024-06-02T15:37:10.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-06-02T15:37:10.397+0000 7f89acfb2700 1 -- 172.21.15.135:0/83205009 wait complete. 2024-06-02T15:37:11.033 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:10 smithi135 ceph-mon[36839]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:11.034 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:11.034 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:11.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:10 smithi120 ceph-mon[36451]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:11.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:11.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:11.062 DEBUG:teuthology.orchestra.run.smithi090:mds.a> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.a.service 2024-06-02T15:37:11.067 DEBUG:teuthology.orchestra.run.smithi090:mds.d> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.d.service 2024-06-02T15:37:11.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:10 smithi090 ceph-mon[31595]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:11.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:11.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:11.079 DEBUG:teuthology.orchestra.run.smithi090:mds.g> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.g.service 2024-06-02T15:37:11.124 DEBUG:teuthology.orchestra.run.smithi090:mds.j> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.j.service 2024-06-02T15:37:11.169 DEBUG:teuthology.orchestra.run.smithi120:mds.b> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.b.service 2024-06-02T15:37:11.171 DEBUG:teuthology.orchestra.run.smithi120:mds.e> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.e.service 2024-06-02T15:37:11.215 DEBUG:teuthology.orchestra.run.smithi120:mds.h> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service 2024-06-02T15:37:11.257 DEBUG:teuthology.orchestra.run.smithi120:mds.k> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.k.service 2024-06-02T15:37:11.300 DEBUG:teuthology.orchestra.run.smithi135:mds.c> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.c.service 2024-06-02T15:37:11.303 DEBUG:teuthology.orchestra.run.smithi135:mds.f> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.f.service 2024-06-02T15:37:11.345 DEBUG:teuthology.orchestra.run.smithi135:mds.i> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.i.service 2024-06-02T15:37:11.388 DEBUG:teuthology.orchestra.run.smithi135:mds.l> sudo journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.l.service 2024-06-02T15:37:11.431 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-02T15:37:11.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:11.837 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:12.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:12.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:12.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:13.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:12 smithi090 ceph-mon[31595]: Deploying daemon mds.l on smithi135 2024-06-02T15:37:13.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:12 smithi090 ceph-mon[31595]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:13.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:12 smithi120 ceph-mon[36451]: Deploying daemon mds.l on smithi135 2024-06-02T15:37:13.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:12 smithi120 ceph-mon[36451]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:13.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:12 smithi135 ceph-mon[36839]: Deploying daemon mds.l on smithi135 2024-06-02T15:37:13.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:12 smithi135 ceph-mon[36839]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:14.067 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:13 smithi135 systemd[1]: Starting Ceph mds.l for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:14.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 -- 172.21.15.90:0/3578869054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030106030 msgr2=0x7f3030112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 --2- 172.21.15.90:0/3578869054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030106030 0x7f3030112540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3028009f90 tx=0x7f302800b330 comp rx=0 tx=0).stop 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 -- 172.21.15.90:0/3578869054 shutdown_connections 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 --2- 172.21.15.90:0/3578869054 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030106030 0x7f3030112540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 --2- 172.21.15.90:0/3578869054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f303010b670 0x7f303010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 --2- 172.21.15.90:0/3578869054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3030105670 0x7f3030105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 -- 172.21.15.90:0/3578869054 >> 172.21.15.90:0/3578869054 conn(0x7f3030101160 msgr2=0x7f3030103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 -- 172.21.15.90:0/3578869054 shutdown_connections 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.100+0000 7f302fe00700 1 -- 172.21.15.90:0/3578869054 wait complete. 2024-06-02T15:37:14.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.101+0000 7f302fe00700 1 Processor -- start 2024-06-02T15:37:14.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 -- start start 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3030106030 0x7f30301a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3030115870 con 0x7f3030106030 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30301156f0 con 0x7f3030105670 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302fe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30301159f0 con 0x7f303010b670 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302ea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3030106030 0x7f30301a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302ea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43230/0 (socket says 172.21.15.90:43230) 2024-06-02T15:37:14.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44414/0 (socket says 172.21.15.90:44414) 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.102+0000 7f302ea00700 1 -- 172.21.15.90:0/2330987312 learned_addr learned my addr 172.21.15.90:0/2330987312 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 -- 172.21.15.90:0/2330987312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 msgr2=0x7f303019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 -- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3030106030 msgr2=0x7f30301a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3030106030 0x7f30301a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 -- 172.21.15.90:0/2330987312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303019c210 con 0x7f3030105670 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302f400700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302ea00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f302000e9c0 tx=0x7f302000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020004a30 con 0x7f3030105670 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303019c500 con 0x7f3030105670 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302000cc70 con 0x7f3030105670 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30200054a0 con 0x7f3030105670 2024-06-02T15:37:14.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.103+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30301a87c0 con 0x7f3030105670 2024-06-02T15:37:14.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.104+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303006a230 con 0x7f3030105670 2024-06-02T15:37:14.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.106+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3020004bd0 con 0x7f3030105670 2024-06-02T15:37:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.108+0000 7f3027400700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 0x7f30140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.108+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f30200e1dc0 con 0x7f3030105670 2024-06-02T15:37:14.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.108+0000 7f302e000700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 0x7f30140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:14.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.109+0000 7f302e000700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 0x7f30140b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f303010ac90 tx=0x7f301c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:14.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.110+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30200a9ee0 con 0x7f3030105670 2024-06-02T15:37:14.332 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 podman[66504]: 2024-06-02 15:37:13.981621759 +0000 UTC m=+0.017376171 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:14.332 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 podman[66504]: 2024-06-02 15:37:14.09958352 +0000 UTC m=+0.135337925 container create 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-02T15:37:14.332 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 podman[66504]: 2024-06-02 15:37:14.335447101 +0000 UTC m=+0.371201495 container init 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-02T15:37:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.392+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f303019cfe0 con 0x7f3030105670 2024-06-02T15:37:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.394+0000 7f3027400700 1 -- 172.21.15.90:0/2330987312 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f30200a97e0 con 0x7f3030105670 2024-06-02T15:37:14.391 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:14.391 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 msgr2=0x7f30140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 0x7f30140b4dc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f303010ac90 tx=0x7f301c009260 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 msgr2=0x7f30301a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f302000e9c0 tx=0x7f302000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 shutdown_connections 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30140b2900 0x7f30140b4dc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f303010b670 0x7f303019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3030105670 0x7f30301a18f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 --2- 172.21.15.90:0/2330987312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3030106030 0x7f30301a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 >> 172.21.15.90:0/2330987312 conn(0x7f3030101160 msgr2=0x7f30301016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 shutdown_connections 2024-06-02T15:37:14.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:14.397+0000 7f302fe00700 1 -- 172.21.15.90:0/2330987312 wait complete. 2024-06-02T15:37:14.396 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 2 2024-06-02T15:37:14.750 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 podman[66504]: 2024-06-02 15:37:14.33807514 +0000 UTC m=+0.373829535 container start 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-02T15:37:14.750 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mds[66523]: -- 172.21.15.135:0/4255251795 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581cca2fa40 con 0x5581cca02800 2024-06-02T15:37:14.750 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l[66519]: starting mds.l at 2024-06-02T15:37:14.750 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 bash[66504]: 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed 2024-06-02T15:37:14.750 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 15:37:14 smithi135 systemd[1]: Started Ceph mds.l for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6832/3349102106,v1:172.21.15.135:6833/3349102106] up:boot 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: fsmap 1 up:standby 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2330987312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6832/3349102106,v1:172.21.15.135:6833/3349102106] up:boot 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: fsmap 1 up:standby 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2330987312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:15.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:15.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6832/3349102106,v1:172.21.15.135:6833/3349102106] up:boot 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: fsmap 1 up:standby 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2330987312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:16.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:15 smithi120 ceph-mon[36451]: Deploying daemon mds.k on smithi120 2024-06-02T15:37:16.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:15 smithi135 ceph-mon[36839]: Deploying daemon mds.k on smithi120 2024-06-02T15:37:16.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:15 smithi090 ceph-mon[31595]: Deploying daemon mds.k on smithi120 2024-06-02T15:37:16.997 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:16 smithi120 ceph-mon[36451]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:16.997 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:16 smithi120 systemd[1]: Starting Ceph mds.k for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:17.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:16 smithi135 ceph-mon[36839]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:17.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:16 smithi090 ceph-mon[31595]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:17.227 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:17.258 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 podman[65722]: 2024-06-02 15:37:16.902181707 +0000 UTC m=+0.017382050 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:17.259 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 podman[65722]: 2024-06-02 15:37:17.034424307 +0000 UTC m=+0.149624619 container create 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 podman[65722]: 2024-06-02 15:37:17.261851568 +0000 UTC m=+0.377051885 container init 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 podman[65722]: 2024-06-02 15:37:17.265006691 +0000 UTC m=+0.380207017 container start 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 ceph-mds[65741]: -- 172.21.15.120:0/33950289 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d0f9399a40 con 0x55d0f936c800 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k[65737]: starting mds.k at 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 bash[65722]: 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d 2024-06-02T15:37:17.558 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 15:37:17 smithi120 systemd[1]: Started Ceph mds.k for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:17.672 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: Deploying daemon mds.j on smithi090 2024-06-02T15:37:18.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:18.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6832/3322691613,v1:172.21.15.120:6833/3322691613] up:boot 2024-06-02T15:37:18.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: fsmap 2 up:standby 2024-06-02T15:37:18.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T15:37:18.684 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.684 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.684 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.684 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: Deploying daemon mds.j on smithi090 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6832/3322691613,v1:172.21.15.120:6833/3322691613] up:boot 2024-06-02T15:37:18.685 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: fsmap 2 up:standby 2024-06-02T15:37:18.686 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T15:37:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: Deploying daemon mds.j on smithi090 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6832/3322691613,v1:172.21.15.120:6833/3322691613] up:boot 2024-06-02T15:37:18.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: fsmap 2 up:standby 2024-06-02T15:37:18.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T15:37:20.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 -- 172.21.15.90:0/3707064664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 msgr2=0x7f3e8c112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:20.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3707064664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c112540 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009f90 tx=0x7f3e8400b330 comp rx=0 tx=0).stop 2024-06-02T15:37:20.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 -- 172.21.15.90:0/3707064664 shutdown_connections 2024-06-02T15:37:20.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3707064664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c112540 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3707064664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3707064664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 0x7f3e8c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 -- 172.21.15.90:0/3707064664 >> 172.21.15.90:0/3707064664 conn(0x7f3e8c101160 msgr2=0x7f3e8c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:20.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.020+0000 7f3e90a00700 1 -- 172.21.15.90:0/3707064664 shutdown_connections 2024-06-02T15:37:20.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.021+0000 7f3e90a00700 1 -- 172.21.15.90:0/3707064664 wait complete. 2024-06-02T15:37:20.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.022+0000 7f3e90a00700 1 Processor -- start 2024-06-02T15:37:20.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.022+0000 7f3e90a00700 1 -- start start 2024-06-02T15:37:20.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 0x7f3e8c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:20.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c1a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:20.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:20.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c1158b0 con 0x7f3e8c105670 2024-06-02T15:37:20.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c115730 con 0x7f3e8c106030 2024-06-02T15:37:20.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e90a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8c115a30 con 0x7f3e8c10b670 2024-06-02T15:37:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e8be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e8be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44442/0 (socket says 172.21.15.90:44442) 2024-06-02T15:37:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e8be00700 1 -- 172.21.15.90:0/3281106024 learned_addr learned my addr 172.21.15.90:0/3281106024 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e8b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 0x7f3e8c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:20.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.023+0000 7f3e8aa00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c1a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 -- 172.21.15.90:0/3281106024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 msgr2=0x7f3e8c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 -- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 msgr2=0x7f3e8c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 0x7f3e8c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 -- 172.21.15.90:0/3281106024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8c19c210 con 0x7f3e8c106030 2024-06-02T15:37:20.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8be00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:37:20.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e8aa00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c1a1e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9c0 tx=0x7f3e7800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:20.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800c9c0 con 0x7f3e8c106030 2024-06-02T15:37:20.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7800cb60 con 0x7f3e8c106030 2024-06-02T15:37:20.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.024+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8c19c500 con 0x7f3e8c106030 2024-06-02T15:37:20.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.025+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78018420 con 0x7f3e8c106030 2024-06-02T15:37:20.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.025+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8c1a87c0 con 0x7f3e8c106030 2024-06-02T15:37:20.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.026+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8c06a230 con 0x7f3e8c106030 2024-06-02T15:37:20.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.028+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3e78010430 con 0x7f3e8c106030 2024-06-02T15:37:20.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.030+0000 7f3e83e00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:20.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.030+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f3e780e1f00 con 0x7f3e8c106030 2024-06-02T15:37:20.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.030+0000 7f3e8b400700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:20.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.031+0000 7f3e8b400700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 0x7f3e700b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c19cbf0 tx=0x7f3e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:20.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.034+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e780aa020 con 0x7f3e8c106030 2024-06-02T15:37:20.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.315+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e8c02e560 con 0x7f3e8c106030 2024-06-02T15:37:20.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.316+0000 7f3e83e00700 1 -- 172.21.15.90:0/3281106024 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f3e780a9920 con 0x7f3e8c106030 2024-06-02T15:37:20.314 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 msgr2=0x7f3e700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 0x7f3e700b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c19cbf0 tx=0x7f3e7c009260 comp rx=0 tx=0).stop 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 msgr2=0x7f3e8c1a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c1a1e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9c0 tx=0x7f3e7800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 shutdown_connections 2024-06-02T15:37:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3e700b2900 0x7f3e700b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e8c10b670 0x7f3e8c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e8c105670 0x7f3e8c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 --2- 172.21.15.90:0/3281106024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3e8c106030 0x7f3e8c1a1e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.319+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 >> 172.21.15.90:0/3281106024 conn(0x7f3e8c101160 msgr2=0x7f3e8c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.320+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 shutdown_connections 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:20.320+0000 7f3e90a00700 1 -- 172.21.15.90:0/3281106024 wait complete. 2024-06-02T15:37:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-06-02T15:37:21.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:20 smithi120 ceph-mon[36451]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:21.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3281106024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:21.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:20 smithi135 ceph-mon[36839]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:21.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3281106024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:21.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:20 smithi090 ceph-mon[31595]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:21.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3281106024' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:22.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:22 smithi090 ceph-mon[31595]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:22 smithi120 ceph-mon[36451]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:23.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:22 smithi135 ceph-mon[36839]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:23.124 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:22 smithi090 podman[71776]: 2024-06-02 15:37:22.903612387 +0000 UTC m=+0.159001594 container create ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe) 2024-06-02T15:37:23.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:23.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 podman[71776]: 2024-06-02 15:37:23.25024225 +0000 UTC m=+0.505631463 container init ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:37:23.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 podman[71776]: 2024-06-02 15:37:23.260468958 +0000 UTC m=+0.515858134 container start ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-02T15:37:23.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 ceph-mds[71799]: -- 172.21.15.90:0/551481825 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56330b1e9a40 con 0x56330b1bc800 2024-06-02T15:37:23.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j[71791]: starting mds.j at 2024-06-02T15:37:23.624 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 bash[71776]: ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a 2024-06-02T15:37:23.624 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 15:37:23 smithi090 systemd[1]: Started Ceph mds.j for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: Deploying daemon mds.d on smithi090 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6834/661907319,v1:172.21.15.90:6835/661907319] up:boot 2024-06-02T15:37:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: fsmap 3 up:standby 2024-06-02T15:37:24.809 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: Deploying daemon mds.d on smithi090 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6834/661907319,v1:172.21.15.90:6835/661907319] up:boot 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: fsmap 3 up:standby 2024-06-02T15:37:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T15:37:24.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: Deploying daemon mds.d on smithi090 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6834/661907319,v1:172.21.15.90:6835/661907319] up:boot 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: fsmap 3 up:standby 2024-06-02T15:37:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T15:37:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 -- 172.21.15.90:0/169380677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406b220 msgr2=0x7f37c406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 --2- 172.21.15.90:0/169380677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406b220 0x7f37c406b6a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37b00098f0 tx=0x7f37b0009c00 comp rx=0 tx=0).stop 2024-06-02T15:37:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 -- 172.21.15.90:0/169380677 shutdown_connections 2024-06-02T15:37:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 --2- 172.21.15.90:0/169380677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406bbe0 0x7f37c41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 --2- 172.21.15.90:0/169380677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 0x7f37c4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 --2- 172.21.15.90:0/169380677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406b220 0x7f37c406b6a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 -- 172.21.15.90:0/169380677 >> 172.21.15.90:0/169380677 conn(0x7f37c40793f0 msgr2=0x7f37c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 -- 172.21.15.90:0/169380677 shutdown_connections 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.916+0000 7f37cac00700 1 -- 172.21.15.90:0/169380677 wait complete. 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.917+0000 7f37cac00700 1 Processor -- start 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.917+0000 7f37cac00700 1 -- start start 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 0x7f37c4102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 0x7f37c4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4113710 con 0x7f37c406b220 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4113590 con 0x7f37c41095b0 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37cac00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4113890 con 0x7f37c406bbe0 2024-06-02T15:37:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 0x7f37c4102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:50324/0 (socket says 172.21.15.90:50324) 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3400700 1 -- 172.21.15.90:0/515763818 learned_addr learned my addr 172.21.15.90:0/515763818 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 0x7f37c4102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48838/0 (socket says 172.21.15.90:48838) 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c8c00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 0x7f37c4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3400700 1 -- 172.21.15.90:0/515763818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 msgr2=0x7f37c4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.918+0000 7f37c3400700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 0x7f37c4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c3400700 1 -- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 msgr2=0x7f37c4102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c3400700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 0x7f37c4102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c3400700 1 -- 172.21.15.90:0/515763818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c41a81f0 con 0x7f37c406bbe0 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c3400700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009230 tx=0x7f37b000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:26.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b00050c0 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b000b560 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c41a8480 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0003a00 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.919+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a88d0 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.921+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c406a230 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.923+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f37b000b710 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.925+0000 7f37c0c00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 0x7f37a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.925+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f37b00e2cf0 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.925+0000 7f37c3e00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 0x7f37a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.926+0000 7f37c3e00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 0x7f37a80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009be0 tx=0x7f37b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:25.929+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37b0013030 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.218+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37c402e560 con 0x7f37c406bbe0 2024-06-02T15:37:26.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.219+0000 7f37c0c00700 1 -- 172.21.15.90:0/515763818 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f37b00aaea0 con 0x7f37c406bbe0 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 msgr2=0x7f37a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 0x7f37a80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009be0 tx=0x7f37b4009260 comp rx=0 tx=0).stop 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 msgr2=0x7f37c4102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:26.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009230 tx=0x7f37b000fa70 comp rx=0 tx=0).stop 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 shutdown_connections 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f37a80b2880 0x7f37a80b4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37c41095b0 0x7f37c4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37c406b220 0x7f37c4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 --2- 172.21.15.90:0/515763818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37c406bbe0 0x7f37c4102a50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.222+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 >> 172.21.15.90:0/515763818 conn(0x7f37c40793f0 msgr2=0x7f37c41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.223+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 shutdown_connections 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:26.223+0000 7f37cac00700 1 -- 172.21.15.90:0/515763818 wait complete. 2024-06-02T15:37:26.394 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-06-02T15:37:27.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:26 smithi120 ceph-mon[36451]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:27.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:26 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/515763818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:27.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:26 smithi135 ceph-mon[36839]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:27.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:26 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/515763818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:26 smithi090 ceph-mon[31595]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:26 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/515763818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:28 smithi090 ceph-mon[31595]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:29.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:28 smithi120 ceph-mon[36451]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:28 smithi135 ceph-mon[36839]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:29.185 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:28 smithi090 podman[72230]: 2024-06-02 15:37:28.907567744 +0000 UTC m=+0.156042631 container create 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-02T15:37:29.186 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 podman[72230]: 2024-06-02 15:37:29.187100357 +0000 UTC m=+0.435575287 container init 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-02T15:37:29.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:29.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 podman[72230]: 2024-06-02 15:37:29.1974619 +0000 UTC m=+0.445936759 container start 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:37:29.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 ceph-mds[72249]: -- 172.21.15.90:0/4257025938 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab57ec7a40 con 0x55ab57e9b000 2024-06-02T15:37:29.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d[72245]: starting mds.d at 2024-06-02T15:37:29.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 bash[72230]: 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 2024-06-02T15:37:29.624 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 15:37:29 smithi090 systemd[1]: Started Ceph mds.d for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:29.646 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: Deploying daemon mds.e on smithi120 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:30.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6836/683051134,v1:172.21.15.90:6837/683051134] up:boot 2024-06-02T15:37:30.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: fsmap 4 up:standby 2024-06-02T15:37:30.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:30 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T15:37:30.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: Deploying daemon mds.e on smithi120 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6836/683051134,v1:172.21.15.90:6837/683051134] up:boot 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: fsmap 4 up:standby 2024-06-02T15:37:30.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:30 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: Deploying daemon mds.e on smithi120 2024-06-02T15:37:30.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:30.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6836/683051134,v1:172.21.15.90:6837/683051134] up:boot 2024-06-02T15:37:30.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: fsmap 4 up:standby 2024-06-02T15:37:30.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:30 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T15:37:31.808 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:31 smithi120 systemd[1]: Starting Ceph mds.e for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:31.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 -- 172.21.15.90:0/3250857662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc10fd00 msgr2=0x7fd0cc112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 --2- 172.21.15.90:0/3250857662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc112130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009f90 tx=0x7fd0c400b330 comp rx=0 tx=0).stop 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 -- 172.21.15.90:0/3250857662 shutdown_connections 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 --2- 172.21.15.90:0/3250857662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc112130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 --2- 172.21.15.90:0/3250857662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc104400 0x7fd0cc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 --2- 172.21.15.90:0/3250857662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc06c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 -- 172.21.15.90:0/3250857662 >> 172.21.15.90:0/3250857662 conn(0x7fd0cc079280 msgr2=0x7fd0cc079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 -- 172.21.15.90:0/3250857662 shutdown_connections 2024-06-02T15:37:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.894+0000 7fd0d3000700 1 -- 172.21.15.90:0/3250857662 wait complete. 2024-06-02T15:37:31.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 Processor -- start 2024-06-02T15:37:31.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 -- start start 2024-06-02T15:37:31.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:31.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 0x7fd0cc1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc115810 con 0x7fd0cc10fd00 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc115690 con 0x7fd0cc06bff0 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.896+0000 7fd0d3000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc115990 con 0x7fd0cc104400 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d2600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 0x7fd0cc1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d2600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53140/0 (socket says 172.21.15.90:53140) 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d2600700 1 -- 172.21.15.90:0/1705196255 learned_addr learned my addr 172.21.15.90:0/1705196255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 -- 172.21.15.90:0/1705196255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 msgr2=0x7fd0cc1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 -- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 msgr2=0x7fd0cc19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1200700 1 -- 172.21.15.90:0/1705196255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc19c1b0 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d1c00700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.897+0000 7fd0d2600700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0d1200700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 0x7fd0cc1a1dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e9f0 tx=0x7fd0b800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800cb70 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b800ee80 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc19c4a0 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8003d40 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.898+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a8760 con 0x7fd0cc104400 2024-06-02T15:37:31.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.899+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0cc06a230 con 0x7fd0cc104400 2024-06-02T15:37:31.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.901+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd0b8003ee0 con 0x7fd0cc104400 2024-06-02T15:37:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.903+0000 7fd0ca000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 0x7fd0b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.903+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd0b80e1f00 con 0x7fd0cc104400 2024-06-02T15:37:31.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.903+0000 7fd0d1c00700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 0x7fd0b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:31.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.904+0000 7fd0d1c00700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 0x7fd0b40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc19cb90 tx=0x7fd0c0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:31.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:31.907+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0b80aa020 con 0x7fd0cc104400 2024-06-02T15:37:32.146 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:31 smithi120 podman[66041]: 2024-06-02 15:37:31.761186198 +0000 UTC m=+0.016985574 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:32.146 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:31 smithi120 podman[66041]: 2024-06-02 15:37:31.905282886 +0000 UTC m=+0.161082283 container create 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:37:32.146 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 podman[66041]: 2024-06-02 15:37:32.149448923 +0000 UTC m=+0.405248297 container init 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:37:32.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.190+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0cc1a89f0 con 0x7fd0cc104400 2024-06-02T15:37:32.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.191+0000 7fd0ca000700 1 -- 172.21.15.90:0/1705196255 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fd0b80a9920 con 0x7fd0cc104400 2024-06-02T15:37:32.189 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:32.189 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 msgr2=0x7fd0b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 0x7fd0b40b4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc19cb90 tx=0x7fd0c0009310 comp rx=0 tx=0).stop 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 msgr2=0x7fd0cc1a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 0x7fd0cc1a1dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e9f0 tx=0x7fd0b800ed00 comp rx=0 tx=0).stop 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 shutdown_connections 2024-06-02T15:37:32.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd0b40b2880 0x7fd0b40b4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0cc10fd00 0x7fd0cc19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd0cc06bff0 0x7fd0cc1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 --2- 172.21.15.90:0/1705196255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0cc104400 0x7fd0cc1a1dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.195+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 >> 172.21.15.90:0/1705196255 conn(0x7fd0cc079280 msgr2=0x7fd0cc1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.196+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 shutdown_connections 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:32.196+0000 7fd0d3000700 1 -- 172.21.15.90:0/1705196255 wait complete. 2024-06-02T15:37:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 5 2024-06-02T15:37:32.557 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 podman[66041]: 2024-06-02 15:37:32.152180606 +0000 UTC m=+0.407979989 container start 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:37:32.557 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mds[66060]: -- 172.21.15.120:0/3739295919 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd71eb1a40 con 0x55dd71e85000 2024-06-02T15:37:32.558 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: starting mds.e at 2024-06-02T15:37:32.558 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 bash[66041]: 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 2024-06-02T15:37:32.558 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:37:32 smithi120 systemd[1]: Started Ceph mds.e for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1705196255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:32 smithi090 ceph-mon[31595]: Deploying daemon mds.f on smithi135 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1705196255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:32.988 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:32.989 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:32.989 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:32.989 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:32 smithi135 ceph-mon[36839]: Deploying daemon mds.f on smithi135 2024-06-02T15:37:33.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:33.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1705196255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:32 smithi120 ceph-mon[36451]: Deploying daemon mds.f on smithi135 2024-06-02T15:37:34.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:33 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6834/3465448684,v1:172.21.15.120:6835/3465448684] up:boot 2024-06-02T15:37:34.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:33 smithi120 ceph-mon[36451]: fsmap 5 up:standby 2024-06-02T15:37:34.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:37:34.070 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:33 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6834/3465448684,v1:172.21.15.120:6835/3465448684] up:boot 2024-06-02T15:37:34.070 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:33 smithi135 ceph-mon[36839]: fsmap 5 up:standby 2024-06-02T15:37:34.070 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:37:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:33 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6834/3465448684,v1:172.21.15.120:6835/3465448684] up:boot 2024-06-02T15:37:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:33 smithi090 ceph-mon[31595]: fsmap 5 up:standby 2024-06-02T15:37:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:37:34.742 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:34 smithi135 systemd[1]: Starting Ceph mds.f for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:35.013 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:34 smithi135 ceph-mon[36839]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:35.014 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:34 smithi135 podman[66808]: 2024-06-02 15:37:34.667340924 +0000 UTC m=+0.017217518 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:35.014 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:34 smithi135 podman[66808]: 2024-06-02 15:37:34.788054075 +0000 UTC m=+0.137930737 container create e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:37:35.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:34 smithi120 ceph-mon[36451]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:35.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:35.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:34 smithi090 ceph-mon[31595]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 podman[66808]: 2024-06-02 15:37:35.016782462 +0000 UTC m=+0.366659055 container init e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 podman[66808]: 2024-06-02 15:37:35.0195085 +0000 UTC m=+0.369385104 container start e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f, RELEASE=main-b7328fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True) 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 ceph-mds[66827]: -- 172.21.15.135:0/2641386090 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b32c929a40 con 0x55b32c8fcc00 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f[66823]: starting mds.f at 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 bash[66808]: e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 2024-06-02T15:37:35.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 15:37:35 smithi135 systemd[1]: Started Ceph mds.f for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:35.518 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:36.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: Deploying daemon mds.c on smithi135 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6834/2238315985,v1:172.21.15.135:6835/2238315985] up:boot 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: fsmap 6 up:standby 2024-06-02T15:37:36.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T15:37:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: Deploying daemon mds.c on smithi135 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6834/2238315985,v1:172.21.15.135:6835/2238315985] up:boot 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: fsmap 6 up:standby 2024-06-02T15:37:36.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: Deploying daemon mds.c on smithi135 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6834/2238315985,v1:172.21.15.135:6835/2238315985] up:boot 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: fsmap 6 up:standby 2024-06-02T15:37:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T15:37:37.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:37 smithi090 ceph-mon[31595]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:37 smithi120 ceph-mon[36451]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:37.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:37 smithi135 ceph-mon[36839]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:37.567 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 systemd[1]: Starting Ceph mds.c for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:37.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 -- 172.21.15.90:0/1778868630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 msgr2=0x7f3634112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:37.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 --2- 172.21.15.90:0/1778868630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f3634112540 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f362c009990 tx=0x7f362c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 -- 172.21.15.90:0/1778868630 shutdown_connections 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 --2- 172.21.15.90:0/1778868630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f3634112540 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 --2- 172.21.15.90:0/1778868630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 0x7f363410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 --2- 172.21.15.90:0/1778868630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f3634105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.777+0000 7f3633e00700 1 -- 172.21.15.90:0/1778868630 >> 172.21.15.90:0/1778868630 conn(0x7f3634101160 msgr2=0x7f3634103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.778+0000 7f3633e00700 1 -- 172.21.15.90:0/1778868630 shutdown_connections 2024-06-02T15:37:37.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.778+0000 7f3633e00700 1 -- 172.21.15.90:0/1778868630 wait complete. 2024-06-02T15:37:37.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 Processor -- start 2024-06-02T15:37:37.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 -- start start 2024-06-02T15:37:37.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:37.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f36341a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 0x7f363419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36341158a0 con 0x7f3634106030 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3634115720 con 0x7f363410b670 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.779+0000 7f3633e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3634115a20 con 0x7f3634105670 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f362a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f36341a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3633400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 0x7f363419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f362a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f36341a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53156/0 (socket says 172.21.15.90:53156) 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38908/0 (socket says 172.21.15.90:38908) 2024-06-02T15:37:37.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f362a000700 1 -- 172.21.15.90:0/4251389526 learned_addr learned my addr 172.21.15.90:0/4251389526 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 -- 172.21.15.90:0/4251389526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 msgr2=0x7f363419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 0x7f363419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 -- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 msgr2=0x7f36341a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f36341a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.780+0000 7f3632a00700 1 -- 172.21.15.90:0/4251389526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363419c400 con 0x7f3634105670 2024-06-02T15:37:37.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f3632a00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f362400ef10 tx=0x7f362400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362400c9c0 con 0x7f3634105670 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363419c690 con 0x7f3634105670 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341a89d0 con 0x7f3634105670 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3624019070 con 0x7f3634105670 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.781+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624014480 con 0x7f3634105670 2024-06-02T15:37:37.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.782+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3634106db0 con 0x7f3634105670 2024-06-02T15:37:37.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.784+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3624012050 con 0x7f3634105670 2024-06-02T15:37:37.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.786+0000 7f362be00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 0x7f36180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:37.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.786+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f36240e1cb0 con 0x7f3634105670 2024-06-02T15:37:37.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.786+0000 7f362a000700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 0x7f36180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:37.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.787+0000 7f362a000700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 0x7f36180b4dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f363419d270 tx=0x7f361c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:37.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:37.788+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3624010040 con 0x7f3634105670 2024-06-02T15:37:37.962 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 podman[67114]: 2024-06-02 15:37:37.600813507 +0000 UTC m=+0.017535105 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:37.962 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 podman[67114]: 2024-06-02 15:37:37.720673342 +0000 UTC m=+0.137394930 container create ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:37:37.963 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 podman[67114]: 2024-06-02 15:37:37.965616356 +0000 UTC m=+0.382337941 container init ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:37:38.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.069+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363419cfe0 con 0x7f3634105670 2024-06-02T15:37:38.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.070+0000 7f362be00700 1 -- 172.21.15.90:0/4251389526 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7f36240a9dd0 con 0x7f3634105670 2024-06-02T15:37:38.068 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:38.068 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-02T15:37:38.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 msgr2=0x7f36180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:38.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 0x7f36180b4dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f363419d270 tx=0x7f361c009260 comp rx=0 tx=0).stop 2024-06-02T15:37:38.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 msgr2=0x7f36341a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:38.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f362400ef10 tx=0x7f362400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:37:38.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 shutdown_connections 2024-06-02T15:37:38.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36180b2900 0x7f36180b4dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f363410b670 0x7f363419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3634105670 0x7f36341a1a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 --2- 172.21.15.90:0/4251389526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3634106030 0x7f36341a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 >> 172.21.15.90:0/4251389526 conn(0x7f3634101160 msgr2=0x7f36341016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.073+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 shutdown_connections 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:38.074+0000 7f3633e00700 1 -- 172.21.15.90:0/4251389526 wait complete. 2024-06-02T15:37:38.072 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 7 2024-06-02T15:37:38.245 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 podman[67114]: 2024-06-02 15:37:37.968403269 +0000 UTC m=+0.385124855 container start ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-02T15:37:38.245 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 ceph-mds[67133]: -- 172.21.15.135:0/335152663 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e28519a40 con 0x564e284ed000 2024-06-02T15:37:38.245 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: starting mds.c at 2024-06-02T15:37:38.245 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:38 smithi135 bash[67114]: ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 2024-06-02T15:37:38.245 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:37:38 smithi135 systemd[1]: Started Ceph mds.c for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4251389526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4251389526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:38.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4251389526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:38.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:39 smithi120 ceph-mon[36451]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:39.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:39 smithi120 ceph-mon[36451]: Deploying daemon mds.h on smithi120 2024-06-02T15:37:39.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:39 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6836/284781637,v1:172.21.15.135:6837/284781637] up:boot 2024-06-02T15:37:39.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:39 smithi120 ceph-mon[36451]: fsmap 7 up:standby 2024-06-02T15:37:39.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:37:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:39 smithi135 ceph-mon[36839]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:39 smithi135 ceph-mon[36839]: Deploying daemon mds.h on smithi120 2024-06-02T15:37:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:39 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6836/284781637,v1:172.21.15.135:6837/284781637] up:boot 2024-06-02T15:37:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:39 smithi135 ceph-mon[36839]: fsmap 7 up:standby 2024-06-02T15:37:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:37:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:39 smithi090 ceph-mon[31595]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:39 smithi090 ceph-mon[31595]: Deploying daemon mds.h on smithi120 2024-06-02T15:37:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:39 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6836/284781637,v1:172.21.15.135:6837/284781637] up:boot 2024-06-02T15:37:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:39 smithi090 ceph-mon[31595]: fsmap 7 up:standby 2024-06-02T15:37:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:37:40.558 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 systemd[1]: Starting Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:40.867 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 podman[66341]: 2024-06-02 15:37:40.498101738 +0000 UTC m=+0.017359225 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:40.868 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 podman[66341]: 2024-06-02 15:37:40.6098187 +0000 UTC m=+0.129076173 container create 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-06-02T15:37:40.868 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 podman[66341]: 2024-06-02 15:37:40.871014056 +0000 UTC m=+0.390271522 container init 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:37:40.977 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:41.252 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 podman[66341]: 2024-06-02 15:37:40.873618975 +0000 UTC m=+0.392876448 container start 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-02T15:37:41.252 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 ceph-mds[66360]: -- 172.21.15.120:0/2156757211 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613c5b3da40 con 0x5613c5b11000 2024-06-02T15:37:41.252 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[66356]: starting mds.h at 2024-06-02T15:37:41.252 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 bash[66341]: 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 2024-06-02T15:37:41.252 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:37:40 smithi120 systemd[1]: Started Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:41.439 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:41.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:41.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:42.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:42 smithi120 ceph-mon[36451]: Deploying daemon mds.i on smithi135 2024-06-02T15:37:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:42 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6836/1676077338,v1:172.21.15.120:6837/1676077338] up:boot 2024-06-02T15:37:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:42 smithi120 ceph-mon[36451]: fsmap 8 up:standby 2024-06-02T15:37:42.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:37:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:42 smithi135 ceph-mon[36839]: Deploying daemon mds.i on smithi135 2024-06-02T15:37:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:42 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6836/1676077338,v1:172.21.15.120:6837/1676077338] up:boot 2024-06-02T15:37:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:42 smithi135 ceph-mon[36839]: fsmap 8 up:standby 2024-06-02T15:37:42.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:37:42.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:42 smithi090 ceph-mon[31595]: Deploying daemon mds.i on smithi135 2024-06-02T15:37:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:42 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6836/1676077338,v1:172.21.15.120:6837/1676077338] up:boot 2024-06-02T15:37:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:42 smithi090 ceph-mon[31595]: fsmap 8 up:standby 2024-06-02T15:37:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:37:43.541 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:43 smithi135 ceph-mon[36839]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:43.542 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 systemd[1]: Starting Ceph mds.i for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:43 smithi120 ceph-mon[36451]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:43.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:43 smithi090 ceph-mon[31595]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:43.811 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 podman[67433]: 2024-06-02 15:37:43.446938174 +0000 UTC m=+0.017387726 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:43.811 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 podman[67433]: 2024-06-02 15:37:43.569005028 +0000 UTC m=+0.139454572 container create c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=main-b7328fe) 2024-06-02T15:37:43.811 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 podman[67433]: 2024-06-02 15:37:43.813280176 +0000 UTC m=+0.383729720 container init c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-02T15:37:43.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.888+0000 7fed2b800700 1 -- 172.21.15.90:0/866987113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed2410b760 msgr2=0x7fed2410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.888+0000 7fed2b800700 1 --2- 172.21.15.90:0/866987113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed2410b760 0x7fed2410bb40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed180098f0 tx=0x7fed18009c00 comp rx=0 tx=0).stop 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 -- 172.21.15.90:0/866987113 shutdown_connections 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 --2- 172.21.15.90:0/866987113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed24106120 0x7fed24112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 --2- 172.21.15.90:0/866987113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed2410b760 0x7fed2410bb40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 --2- 172.21.15.90:0/866987113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24105760 0x7fed24105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 -- 172.21.15.90:0/866987113 >> 172.21.15.90:0/866987113 conn(0x7fed24101250 msgr2=0x7fed24103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 -- 172.21.15.90:0/866987113 shutdown_connections 2024-06-02T15:37:43.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.889+0000 7fed2b800700 1 -- 172.21.15.90:0/866987113 wait complete. 2024-06-02T15:37:43.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 Processor -- start 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 -- start start 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 0x7fed2419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2410b760 0x7fed241a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed24115960 con 0x7fed24106120 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.890+0000 7fed2b800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fed241157e0 con 0x7fed24105760 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed2b800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed24115ae0 con 0x7fed2410b760 2024-06-02T15:37:43.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54074/0 (socket says 172.21.15.90:54074) 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 -- 172.21.15.90:0/1219919816 learned_addr learned my addr 172.21.15.90:0/1219919816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed28e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 0x7fed2419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed29800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2410b760 0x7fed241a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 -- 172.21.15.90:0/1219919816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2410b760 msgr2=0x7fed241a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2410b760 0x7fed241a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 -- 172.21.15.90:0/1219919816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 msgr2=0x7fed2419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 0x7fed2419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed23e00700 1 -- 172.21.15.90:0/1219919816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed241a0e60 con 0x7fed24106120 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.891+0000 7fed28e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 0x7fed2419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:37:43.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed23e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e9c0 tx=0x7fed1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1000cbd0 con 0x7fed24106120 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed241a1150 con 0x7fed24106120 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1000cd70 con 0x7fed24106120 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10005550 con 0x7fed24106120 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.892+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a89d0 con 0x7fed24106120 2024-06-02T15:37:43.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.893+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2410dcf0 con 0x7fed24106120 2024-06-02T15:37:43.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.895+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fed10004750 con 0x7fed24106120 2024-06-02T15:37:43.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.897+0000 7fed21600700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 0x7fed0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:43.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.897+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fed100e1fa0 con 0x7fed24106120 2024-06-02T15:37:43.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.897+0000 7fed28e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 0x7fed0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:43.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.898+0000 7fed28e00700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 0x7fed0c0b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fed18009fd0 tx=0x7fed18019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:43.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:43.901+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed100aa0c0 con 0x7fed24106120 2024-06-02T15:37:44.067 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 podman[67433]: 2024-06-02 15:37:43.815962173 +0000 UTC m=+0.386411718 container start c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521) 2024-06-02T15:37:44.067 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 ceph-mds[67452]: -- 172.21.15.135:0/656707348 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb3d35ba40 con 0x55bb3d32ec00 2024-06-02T15:37:44.067 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: starting mds.i at 2024-06-02T15:37:44.067 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 bash[67433]: c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c 2024-06-02T15:37:44.067 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:37:43 smithi135 systemd[1]: Started Ceph mds.i for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:44.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.188+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed2406a230 con 0x7fed24106120 2024-06-02T15:37:44.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.189+0000 7fed21600700 1 -- 172.21.15.90:0/1219919816 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7fed100a99c0 con 0x7fed24106120 2024-06-02T15:37:44.186 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.192+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 msgr2=0x7fed0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.192+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 0x7fed0c0b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fed18009fd0 tx=0x7fed18019040 comp rx=0 tx=0).stop 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.192+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 msgr2=0x7fed2419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.192+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e9c0 tx=0x7fed1000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 shutdown_connections 2024-06-02T15:37:44.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fed0c0b2900 0x7fed0c0b4dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2410b760 0x7fed241a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fed24105760 0x7fed2419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 --2- 172.21.15.90:0/1219919816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed24106120 0x7fed2419c390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 >> 172.21.15.90:0/1219919816 conn(0x7fed24101250 msgr2=0x7fed24101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 shutdown_connections 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:44.193+0000 7fed2b800700 1 -- 172.21.15.90:0/1219919816 wait complete. 2024-06-02T15:37:44.191 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 9 2024-06-02T15:37:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: Deploying daemon mds.g on smithi090 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1219919816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6838/3736093015,v1:172.21.15.135:6839/3736093015] up:boot 2024-06-02T15:37:45.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: fsmap 9 up:standby 2024-06-02T15:37:45.309 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:44 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: Deploying daemon mds.g on smithi090 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1219919816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6838/3736093015,v1:172.21.15.135:6839/3736093015] up:boot 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: fsmap 9 up:standby 2024-06-02T15:37:45.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:44 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: Deploying daemon mds.g on smithi090 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1219919816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:45.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6838/3736093015,v1:172.21.15.135:6839/3736093015] up:boot 2024-06-02T15:37:45.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: fsmap 9 up:standby 2024-06-02T15:37:45.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:44 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:37:47.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:47.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:46 smithi090 ceph-mon[31595]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:47.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:46 smithi120 ceph-mon[36451]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:47.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:46 smithi135 ceph-mon[36839]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:47.595 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:49.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:48 smithi090 ceph-mon[31595]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:49.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:48 smithi120 ceph-mon[36451]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:49.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:48 smithi135 ceph-mon[36839]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:49.591 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 15:37:49 smithi090 systemd[1]: Started Ceph mds.g for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:50.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 -- 172.21.15.90:0/89803656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c10fff0 msgr2=0x7fba2c1123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 --2- 172.21.15.90:0/89803656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c10fff0 0x7fba2c1123e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b0a0 tx=0x7fba2400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 -- 172.21.15.90:0/89803656 shutdown_connections 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 --2- 172.21.15.90:0/89803656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c10fff0 0x7fba2c1123e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 --2- 172.21.15.90:0/89803656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c06b250 0x7fba2c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 --2- 172.21.15.90:0/89803656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 0x7fba2c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 -- 172.21.15.90:0/89803656 >> 172.21.15.90:0/89803656 conn(0x7fba2c0793f0 msgr2=0x7fba2c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 -- 172.21.15.90:0/89803656 shutdown_connections 2024-06-02T15:37:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.062+0000 7fba31800700 1 -- 172.21.15.90:0/89803656 wait complete. 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.063+0000 7fba31800700 1 Processor -- start 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 -- start start 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 0x7fba2c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c10fff0 0x7fba2c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c115960 con 0x7fba2c06b250 2024-06-02T15:37:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1157e0 con 0x7fba2c10fff0 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba31800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c115ae0 con 0x7fba2c06bb70 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54114/0 (socket says 172.21.15.90:54114) 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba2aa00700 1 -- 172.21.15.90:0/4112348051 learned_addr learned my addr 172.21.15.90:0/4112348051 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.064+0000 7fba2a000700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 0x7fba2c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2b400700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c10fff0 0x7fba2c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 -- 172.21.15.90:0/4112348051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 msgr2=0x7fba2c19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 0x7fba2c19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 -- 172.21.15.90:0/4112348051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c10fff0 msgr2=0x7fba2c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c10fff0 0x7fba2c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 -- 172.21.15.90:0/4112348051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c1a0eb0 con 0x7fba2c06b250 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.065+0000 7fba2aa00700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e9c0 tx=0x7fba1c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.066+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00cbd0 con 0x7fba2c06b250 2024-06-02T15:37:50.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.066+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c1a11a0 con 0x7fba2c06b250 2024-06-02T15:37:50.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.066+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba1c00ee60 con 0x7fba2c06b250 2024-06-02T15:37:50.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.066+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0055c0 con 0x7fba2c06b250 2024-06-02T15:37:50.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.066+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c1a8a20 con 0x7fba2c06b250 2024-06-02T15:37:50.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.067+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c06a230 con 0x7fba2c06b250 2024-06-02T15:37:50.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.069+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fba1c004750 con 0x7fba2c06b250 2024-06-02T15:37:50.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.071+0000 7fba23400700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 0x7fba100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:50.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.071+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fba1c0e2300 con 0x7fba2c06b250 2024-06-02T15:37:50.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.071+0000 7fba2a000700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 0x7fba100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:50.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.072+0000 7fba2a000700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 0x7fba100b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba2c103a00 tx=0x7fba18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:50.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.074+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba1c054df0 con 0x7fba2c06b250 2024-06-02T15:37:50.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.358+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba2c19d200 con 0x7fba2c06b250 2024-06-02T15:37:50.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.359+0000 7fba23400700 1 -- 172.21.15.90:0/4112348051 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fba1c0aa420 con 0x7fba2c06b250 2024-06-02T15:37:50.357 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:50.357 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-06-02T15:37:50.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.362+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 msgr2=0x7fba100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:50.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.362+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 0x7fba100b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba2c103a00 tx=0x7fba18009260 comp rx=0 tx=0).stop 2024-06-02T15:37:50.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.362+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 msgr2=0x7fba2c19bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.362+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e9c0 tx=0x7fba1c00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 shutdown_connections 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fba100b2880 0x7fba100b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fba2c10fff0 0x7fba2c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba2c06b250 0x7fba2c19bef0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 --2- 172.21.15.90:0/4112348051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c06bb70 0x7fba2c19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 >> 172.21.15.90:0/4112348051 conn(0x7fba2c0793f0 msgr2=0x7fba2c1114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 shutdown_connections 2024-06-02T15:37:50.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:50.363+0000 7fba31800700 1 -- 172.21.15.90:0/4112348051 wait complete. 2024-06-02T15:37:50.362 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: Deploying daemon mds.a on smithi090 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:boot 2024-06-02T15:37:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: fsmap 10 up:standby 2024-06-02T15:37:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:37:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:50 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4112348051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:51.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: Deploying daemon mds.a on smithi090 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:boot 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: fsmap 10 up:standby 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:37:51.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:50 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4112348051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: Deploying daemon mds.a on smithi090 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:boot 2024-06-02T15:37:51.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: fsmap 10 up:standby 2024-06-02T15:37:51.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:37:51.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:50 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4112348051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:52.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:52 smithi090 ceph-mon[31595]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:53.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:52 smithi120 ceph-mon[36451]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:53.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:52 smithi135 ceph-mon[36839]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:53.190 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:37:53.623 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:53 smithi090 systemd[1]: Starting Ceph mds.a for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:54.082 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:53 smithi090 podman[73501]: 2024-06-02 15:37:53.66403257 +0000 UTC m=+0.035057197 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:54.082 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:53 smithi090 podman[73501]: 2024-06-02 15:37:53.794282788 +0000 UTC m=+0.165307452 container create 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:37:54.082 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 podman[73501]: 2024-06-02 15:37:54.083413556 +0000 UTC m=+0.454438254 container init 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe) 2024-06-02T15:37:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:37:54.355 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 podman[73501]: 2024-06-02 15:37:54.094493866 +0000 UTC m=+0.465518512 container start 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-06-02T15:37:54.355 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 ceph-mds[73540]: -- 172.21.15.90:0/158552987 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563cacf03a40 con 0x563caced6c00 2024-06-02T15:37:54.355 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: starting mds.a at 2024-06-02T15:37:54.355 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 bash[73501]: 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 2024-06-02T15:37:54.355 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:37:54 smithi090 systemd[1]: Started Ceph mds.a for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: Deploying daemon mds.b on smithi120 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:boot 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: fsmap 11 up:standby 2024-06-02T15:37:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:55 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:37:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: Deploying daemon mds.b on smithi120 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:boot 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: fsmap 11 up:standby 2024-06-02T15:37:55.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:55 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:37:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: Deploying daemon mds.b on smithi120 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:boot 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: fsmap 11 up:standby 2024-06-02T15:37:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:55 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 -- 172.21.15.90:0/1056849091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334106010 msgr2=0x7f8334112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 --2- 172.21.15.90:0/1056849091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334106010 0x7f8334112520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f833000b0a0 tx=0x7f833000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 -- 172.21.15.90:0/1056849091 shutdown_connections 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 --2- 172.21.15.90:0/1056849091 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334106010 0x7f8334112520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 --2- 172.21.15.90:0/1056849091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 0x7f833410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 --2- 172.21.15.90:0/1056849091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334105650 0x7f8334105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 -- 172.21.15.90:0/1056849091 >> 172.21.15.90:0/1056849091 conn(0x7f8334101120 msgr2=0x7f8334103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 -- 172.21.15.90:0/1056849091 shutdown_connections 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.649+0000 7f833b200700 1 -- 172.21.15.90:0/1056849091 wait complete. 2024-06-02T15:37:56.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.650+0000 7f833b200700 1 Processor -- start 2024-06-02T15:37:56.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.650+0000 7f833b200700 1 -- start start 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334106010 0x7f83341a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 0x7f833419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8334115880 con 0x7f833410b650 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8334115700 con 0x7f8334105650 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8334115a00 con 0x7f8334106010 2024-06-02T15:37:56.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f8339e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f8339e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53214/0 (socket says 172.21.15.90:53214) 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f8339e00700 1 -- 172.21.15.90:0/3398399665 learned_addr learned my addr 172.21.15.90:0/3398399665 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f833a800700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 0x7f833419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.651+0000 7f8339400700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334106010 0x7f83341a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 -- 172.21.15.90:0/3398399665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334106010 msgr2=0x7f83341a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334106010 0x7f83341a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 -- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 msgr2=0x7f833419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 0x7f833419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 -- 172.21.15.90:0/3398399665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833419c220 con 0x7f8334105650 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f8339e00700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f832800e9a0 tx=0x7f832800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832800ca10 con 0x7f8334105650 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832800cbb0 con 0x7f8334105650 2024-06-02T15:37:56.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328018680 con 0x7f8334105650 2024-06-02T15:37:56.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833419c510 con 0x7f8334105650 2024-06-02T15:37:56.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.652+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a87d0 con 0x7f8334105650 2024-06-02T15:37:56.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.653+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833406a230 con 0x7f8334105650 2024-06-02T15:37:56.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.656+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8328010430 con 0x7f8334105650 2024-06-02T15:37:56.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.657+0000 7f832ea00700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 0x7f831c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:37:56.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.657+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f83280e2200 con 0x7f8334105650 2024-06-02T15:37:56.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.657+0000 7f8339400700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 0x7f831c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:37:56.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.658+0000 7f8339400700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 0x7f831c0b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f833410ac70 tx=0x7f8324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:37:56.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.659+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83280aa320 con 0x7f8334105650 2024-06-02T15:37:56.745 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:56 smithi120 systemd[1]: Starting Ceph mds.b for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:37:56.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.941+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f833402e560 con 0x7f8334105650 2024-06-02T15:37:56.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.942+0000 7f832ea00700 1 -- 172.21.15.90:0/3398399665 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f83280b23a0 con 0x7f8334105650 2024-06-02T15:37:56.940 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:37:56.940 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-06-02T15:37:56.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 msgr2=0x7f831c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:56.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 0x7f831c0b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f833410ac70 tx=0x7f8324009260 comp rx=0 tx=0).stop 2024-06-02T15:37:56.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 msgr2=0x7f83341a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:37:56.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f832800e9a0 tx=0x7f832800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 shutdown_connections 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f831c0b2900 0x7f831c0b4dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f833410b650 0x7f833419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8334105650 0x7f83341a1900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 --2- 172.21.15.90:0/3398399665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8334106010 0x7f83341a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.945+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 >> 172.21.15.90:0/3398399665 conn(0x7f8334101120 msgr2=0x7f83341016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.946+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 shutdown_connections 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:37:56.946+0000 7f833b200700 1 -- 172.21.15.90:0/3398399665 wait complete. 2024-06-02T15:37:56.944 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 12 2024-06-02T15:37:56.997 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:56 smithi120 podman[66655]: 2024-06-02 15:37:56.748092432 +0000 UTC m=+0.106370253 container create 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-06-02T15:37:56.997 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:56 smithi120 podman[66655]: 2024-06-02 15:37:56.658996427 +0000 UTC m=+0.017274261 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:37:56.998 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 podman[66655]: 2024-06-02 15:37:57.000693396 +0000 UTC m=+0.358971237 container init 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-02T15:37:57.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 podman[66655]: 2024-06-02 15:37:57.003554235 +0000 UTC m=+0.361832075 container start 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-02T15:37:57.308 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mds[66675]: -- 172.21.15.120:0/852190849 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf18c39a40 con 0x55bf18c0d000 2024-06-02T15:37:57.308 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: starting mds.b at 2024-06-02T15:37:57.308 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 bash[66655]: 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 2024-06-02T15:37:57.308 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:37:57 smithi120 systemd[1]: Started Ceph mds.b for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3398399665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:57 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3398399665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:57 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:57.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3398399665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:37:57.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:57.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:57 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:58 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:boot 2024-06-02T15:37:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:58 smithi135 ceph-mon[36839]: fsmap 12 up:standby 2024-06-02T15:37:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:37:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:58.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:58 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:boot 2024-06-02T15:37:58.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:58 smithi120 ceph-mon[36451]: fsmap 12 up:standby 2024-06-02T15:37:58.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:37:58.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:58.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:58.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:58 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:boot 2024-06-02T15:37:58.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:58 smithi090 ceph-mon[31595]: fsmap 12 up:standby 2024-06-02T15:37:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:37:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:37:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:59 smithi090 ceph-mon[31595]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:37:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:59 smithi120 ceph-mon[36451]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:59.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:37:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:59 smithi135 ceph-mon[36839]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:37:59.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:37:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:37:59.831 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:38:00.307 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:00.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:00.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:00.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:01.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:01.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:01.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:38:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:38:01.643 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:01.643 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.643 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.643 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:01.644 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:01.644 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:01.644 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.546+0000 7f2c17a00700 1 -- 172.21.15.90:0/736128919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 msgr2=0x7f2c10103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.546+0000 7f2c17a00700 1 --2- 172.21.15.90:0/736128919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c10103a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009990 tx=0x7f2bfc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 -- 172.21.15.90:0/736128919 shutdown_connections 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 --2- 172.21.15.90:0/736128919 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 0x7f2c10110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 --2- 172.21.15.90:0/736128919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 0x7f2c101088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 --2- 172.21.15.90:0/736128919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c10103a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 -- 172.21.15.90:0/736128919 >> 172.21.15.90:0/736128919 conn(0x7f2c1007a6a0 msgr2=0x7f2c1007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:02.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 -- 172.21.15.90:0/736128919 shutdown_connections 2024-06-02T15:38:02.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.547+0000 7f2c17a00700 1 -- 172.21.15.90:0/736128919 wait complete. 2024-06-02T15:38:02.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.548+0000 7f2c17a00700 1 Processor -- start 2024-06-02T15:38:02.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.548+0000 7f2c17a00700 1 -- start start 2024-06-02T15:38:02.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:02.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 0x7f2c1019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:02.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 0x7f2c101a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10113790 con 0x7f2c10108500 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10113610 con 0x7f2c10103f90 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c17a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10113910 con 0x7f2c101035f0 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 0x7f2c101a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60728/0 (socket says 172.21.15.90:60728) 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15000700 1 -- 172.21.15.90:0/1517021049 learned_addr learned my addr 172.21.15.90:0/1517021049 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c0fe00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 0x7f2c1019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15000700 1 -- 172.21.15.90:0/1517021049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 msgr2=0x7f2c1019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:02.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.549+0000 7f2c15000700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 0x7f2c1019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c15000700 1 -- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 msgr2=0x7f2c101a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c15000700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 0x7f2c101a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c15000700 1 -- 172.21.15.90:0/1517021049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c101a0e60 con 0x7f2c101035f0 2024-06-02T15:38:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c15000700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400e970 tx=0x7f2c0400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:02.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04004220 con 0x7f2c101035f0 2024-06-02T15:38:02.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c101a1150 con 0x7f2c101035f0 2024-06-02T15:38:02.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c040043c0 con 0x7f2c101035f0 2024-06-02T15:38:02.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04003ce0 con 0x7f2c101035f0 2024-06-02T15:38:02.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.550+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101a89d0 con 0x7f2c101035f0 2024-06-02T15:38:02.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.552+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1006a230 con 0x7f2c101035f0 2024-06-02T15:38:02.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.555+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2c0400c7e0 con 0x7f2c101035f0 2024-06-02T15:38:02.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.556+0000 7f2c0d600700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 0x7f2bf80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:02.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.557+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2c040e1fb0 con 0x7f2c101035f0 2024-06-02T15:38:02.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.557+0000 7f2c0fe00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 0x7f2bf80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:02.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.558+0000 7f2c0fe00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 0x7f2bf80b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b2e0 tx=0x7f2bfc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:02.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.561+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c040aa0d0 con 0x7f2c101035f0 2024-06-02T15:38:02.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.850+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c1019d1b0 con 0x7f2c101035f0 2024-06-02T15:38:02.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.851+0000 7f2c0d600700 1 -- 172.21.15.90:0/1517021049 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f2c040bee20 con 0x7f2c101035f0 2024-06-02T15:38:02.850 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:02.850 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 msgr2=0x7f2bf80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 0x7f2bf80b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00b2e0 tx=0x7f2bfc005960 comp rx=0 tx=0).stop 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 msgr2=0x7f2c1019be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400e970 tx=0x7f2c0400ec80 comp rx=0 tx=0).stop 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 shutdown_connections 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2bf80b2880 0x7f2bf80b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c10108500 0x7f2c101a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c101035f0 0x7f2c1019be50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 --2- 172.21.15.90:0/1517021049 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c10103f90 0x7f2c1019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:02.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.854+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 >> 172.21.15.90:0/1517021049 conn(0x7f2c1007a6a0 msgr2=0x7f2c10101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:02.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.855+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 shutdown_connections 2024-06-02T15:38:02.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:02.855+0000 7f2c17a00700 1 -- 172.21.15.90:0/1517021049 wait complete. 2024-06-02T15:38:02.854 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 13 2024-06-02T15:38:03.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:03.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:03 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1517021049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:03.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:03 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1517021049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:03 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1517021049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:03.939 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-02T15:38:04.401 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:05.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:05 smithi090 ceph-mon[31595]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:05.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:05 smithi120 ceph-mon[36451]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:05.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:05 smithi135 ceph-mon[36839]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 -- 172.21.15.90:0/1200356171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 msgr2=0x7f8b60112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 --2- 172.21.15.90:0/1200356171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b60112110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009990 tx=0x7f8b58009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 -- 172.21.15.90:0/1200356171 shutdown_connections 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 --2- 172.21.15.90:0/1200356171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b60112110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 --2- 172.21.15.90:0/1200356171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b60075cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 --2- 172.21.15.90:0/1200356171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 -- 172.21.15.90:0/1200356171 >> 172.21.15.90:0/1200356171 conn(0x7f8b600fef90 msgr2=0x7f8b601013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 -- 172.21.15.90:0/1200356171 shutdown_connections 2024-06-02T15:38:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.556+0000 7f8b64c00700 1 -- 172.21.15.90:0/1200356171 wait complete. 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.557+0000 7f8b64c00700 1 Processor -- start 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.557+0000 7f8b64c00700 1 -- start start 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.557+0000 7f8b64c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b6010f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.557+0000 7f8b64c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.557+0000 7f8b64c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b6010ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b64c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60115840 con 0x7f8b60076230 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b64c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b601156c0 con 0x7f8b60075910 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b64c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b601159c0 con 0x7f8b6010fd00 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54556/0 (socket says 172.21.15.90:54556) 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b5f400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b6010f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 -- 172.21.15.90:0/3396559090 learned_addr learned my addr 172.21.15.90:0/3396559090 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b5fe00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b6010ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 -- 172.21.15.90:0/3396559090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 msgr2=0x7f8b6010ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b6010ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 -- 172.21.15.90:0/3396559090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 msgr2=0x7f8b6010f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b6010f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b56a00700 1 -- 172.21.15.90:0/3396559090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601ac580 con 0x7f8b60076230 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.558+0000 7f8b5f400700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b6010f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b56a00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800eb30 tx=0x7f8b4800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800cbd0 con 0x7f8b60076230 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b480044d0 con 0x7f8b60076230 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601ac750 con 0x7f8b60076230 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48003e30 con 0x7f8b60076230 2024-06-02T15:38:06.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.559+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601acc30 con 0x7f8b60076230 2024-06-02T15:38:06.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.560+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b60076fc0 con 0x7f8b60076230 2024-06-02T15:38:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.564+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8b48004000 con 0x7f8b60076230 2024-06-02T15:38:06.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.566+0000 7f8b5cc00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 0x7f8b440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:06.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.566+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8b480e2220 con 0x7f8b60076230 2024-06-02T15:38:06.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.566+0000 7f8b5f400700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 0x7f8b440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:06.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.567+0000 7f8b5f400700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 0x7f8b440b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8b50003e30 tx=0x7f8b5000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:06.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.568+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b48054df0 con 0x7f8b60076230 2024-06-02T15:38:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.819+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8b6010dc70 con 0x7f8b60076230 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.819+0000 7f8b5cc00700 1 -- 172.21.15.90:0/3396559090 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f8b480042f0 con 0x7f8b60076230 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-06-02T15:38:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:38:06.819 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-06-02T15:38:06.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.823+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 msgr2=0x7f8b440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:06.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.823+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 0x7f8b440b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8b50003e30 tx=0x7f8b5000b040 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.823+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 msgr2=0x7f8b6010c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.823+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800eb30 tx=0x7f8b4800ee40 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 shutdown_connections 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b440b2880 0x7f8b440b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b6010fd00 0x7f8b6010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b60075910 0x7f8b6010f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 --2- 172.21.15.90:0/3396559090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b60076230 0x7f8b6010c740 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 >> 172.21.15.90:0/3396559090 conn(0x7f8b600fef90 msgr2=0x7f8b600ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 shutdown_connections 2024-06-02T15:38:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:06.824+0000 7f8b64c00700 1 -- 172.21.15.90:0/3396559090 wait complete. 2024-06-02T15:38:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:07.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:07 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3396559090' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T15:38:07.758 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-02T15:38:07.759 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:07 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3396559090' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T15:38:07.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:38:07.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:07 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3396559090' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T15:38:08.204 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:09.554 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:09 smithi090 ceph-mon[31595]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:09.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:09 smithi120 ceph-mon[36451]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:09.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:09 smithi135 ceph-mon[36839]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 -- 172.21.15.90:0/154934500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f491010ff40 msgr2=0x7f4910112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 --2- 172.21.15.90:0/154934500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f491010ff40 0x7f4910112330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4908009990 tx=0x7f4908009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 -- 172.21.15.90:0/154934500 shutdown_connections 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 --2- 172.21.15.90:0/154934500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f491010ff40 0x7f4910112330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 --2- 172.21.15.90:0/154934500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910075770 0x7f4910075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 --2- 172.21.15.90:0/154934500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4910076090 0x7f491010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.426+0000 7f4915000700 1 -- 172.21.15.90:0/154934500 >> 172.21.15.90:0/154934500 conn(0x7f491007a6a0 msgr2=0x7f491007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.427+0000 7f4915000700 1 -- 172.21.15.90:0/154934500 shutdown_connections 2024-06-02T15:38:10.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.427+0000 7f4915000700 1 -- 172.21.15.90:0/154934500 wait complete. 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 Processor -- start 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 -- start start 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910076090 0x7f491019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f491010ff40 0x7f49101a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49101158d0 con 0x7f4910075770 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115750 con 0x7f491010ff40 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.428+0000 7f4915000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4910115a50 con 0x7f4910076090 2024-06-02T15:38:10.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54570/0 (socket says 172.21.15.90:54570) 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 -- 172.21.15.90:0/184886343 learned_addr learned my addr 172.21.15.90:0/184886343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f4905400700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910076090 0x7f491019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490e800700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f491010ff40 0x7f49101a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 -- 172.21.15.90:0/184886343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910076090 msgr2=0x7f491019c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910076090 0x7f491019c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 -- 172.21.15.90:0/184886343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f491010ff40 msgr2=0x7f49101a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f491010ff40 0x7f49101a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.429+0000 7f490de00700 1 -- 172.21.15.90:0/184886343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101a0da0 con 0x7f4910075770 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f490de00700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f490000ef10 tx=0x7f490000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:10.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490000c9c0 con 0x7f4910075770 2024-06-02T15:38:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4900019070 con 0x7f4910075770 2024-06-02T15:38:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101a1030 con 0x7f4910075770 2024-06-02T15:38:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900014480 con 0x7f4910075770 2024-06-02T15:38:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.430+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101a8910 con 0x7f4910075770 2024-06-02T15:38:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.431+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491006a230 con 0x7f4910075770 2024-06-02T15:38:10.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.435+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4900012050 con 0x7f4910075770 2024-06-02T15:38:10.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.436+0000 7f4907400700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 0x7f48f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:10.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.437+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f49000e2360 con 0x7f4910075770 2024-06-02T15:38:10.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.437+0000 7f4905400700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 0x7f48f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:10.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.437+0000 7f4905400700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 0x7f48f40b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4910102740 tx=0x7f48f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:10.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.439+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4900010040 con 0x7f4910075770 2024-06-02T15:38:10.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:10.674+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f491002e560 con 0x7f4910075770 2024-06-02T15:38:11.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.376+0000 7f4907400700 1 -- 172.21.15.90:0/184886343 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f49000aa480 con 0x7f4910075770 2024-06-02T15:38:11.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.382+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 msgr2=0x7f48f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:11.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.382+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 0x7f48f40b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4910102740 tx=0x7f48f8009260 comp rx=0 tx=0).stop 2024-06-02T15:38:11.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.382+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 msgr2=0x7f491019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:11.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.382+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f490000ef10 tx=0x7f490000c3a0 comp rx=0 tx=0).stop 2024-06-02T15:38:11.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 shutdown_connections 2024-06-02T15:38:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f48f40b2900 0x7f48f40b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f491010ff40 0x7f49101a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4910075770 0x7f491019bde0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 --2- 172.21.15.90:0/184886343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4910076090 0x7f491019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:11.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.383+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 >> 172.21.15.90:0/184886343 conn(0x7f491007a6a0 msgr2=0x7f4910101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:11.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.385+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 shutdown_connections 2024-06-02T15:38:11.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:11.385+0000 7f4915000700 1 -- 172.21.15.90:0/184886343 wait complete. 2024-06-02T15:38:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-06-02T15:38:11.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:11 smithi090 ceph-mon[31595]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:11.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T15:38:11.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:11 smithi120 ceph-mon[36451]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:11.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T15:38:11.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:11 smithi135 ceph-mon[36839]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:11.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T15:38:12.282 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-02T15:38:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:12 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T15:38:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:12 smithi090 ceph-mon[31595]: osdmap e73: 12 total, 12 up, 12 in 2024-06-02T15:38:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:12 smithi090 ceph-mon[31595]: pgmap v310: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:12 smithi090 ceph-mon[31595]: osdmap e74: 12 total, 12 up, 12 in 2024-06-02T15:38:12.730 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:12 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T15:38:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:12 smithi120 ceph-mon[36451]: osdmap e73: 12 total, 12 up, 12 in 2024-06-02T15:38:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:12 smithi120 ceph-mon[36451]: pgmap v310: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:12 smithi120 ceph-mon[36451]: osdmap e74: 12 total, 12 up, 12 in 2024-06-02T15:38:12.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:12 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/184886343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T15:38:12.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:12 smithi135 ceph-mon[36839]: osdmap e73: 12 total, 12 up, 12 in 2024-06-02T15:38:12.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:12 smithi135 ceph-mon[36839]: pgmap v310: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:12.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:12 smithi135 ceph-mon[36839]: osdmap e74: 12 total, 12 up, 12 in 2024-06-02T15:38:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:13 smithi090 ceph-mon[31595]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:13 smithi090 ceph-mon[31595]: osdmap e75: 12 total, 12 up, 12 in 2024-06-02T15:38:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:13 smithi120 ceph-mon[36451]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:13.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:13 smithi120 ceph-mon[36451]: osdmap e75: 12 total, 12 up, 12 in 2024-06-02T15:38:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:13 smithi135 ceph-mon[36839]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:13 smithi135 ceph-mon[36839]: osdmap e75: 12 total, 12 up, 12 in 2024-06-02T15:38:14.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:14 smithi090 ceph-mon[31595]: pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:14 smithi120 ceph-mon[36451]: pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:14.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:14 smithi135 ceph-mon[36839]: pgmap v313: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 -- 172.21.15.90:0/973067347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 msgr2=0x7f7ec010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 --2- 172.21.15.90:0/973067347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec010e7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0099c0 tx=0x7f7eac009cd0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 -- 172.21.15.90:0/973067347 shutdown_connections 2024-06-02T15:38:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 --2- 172.21.15.90:0/973067347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec0116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 --2- 172.21.15.90:0/973067347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 0x7f7ec010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 --2- 172.21.15.90:0/973067347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec010e7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 -- 172.21.15.90:0/973067347 >> 172.21.15.90:0/973067347 conn(0x7f7ec0076d10 msgr2=0x7f7ec0108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:15.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 -- 172.21.15.90:0/973067347 shutdown_connections 2024-06-02T15:38:15.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.097+0000 7f7ec7000700 1 -- 172.21.15.90:0/973067347 wait complete. 2024-06-02T15:38:15.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.098+0000 7f7ec7000700 1 Processor -- start 2024-06-02T15:38:15.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 -- start start 2024-06-02T15:38:15.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 0x7f7ec01a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:15.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec01a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec01a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0119d80 con 0x7f7ec010eeb0 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0119c00 con 0x7f7ec0109dd0 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec7000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0119f00 con 0x7f7ec010a780 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec01a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.099+0000 7f7ec5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec01a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52440/0 (socket says 172.21.15.90:52440) 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ec5000700 1 -- 172.21.15.90:0/959824915 learned_addr learned my addr 172.21.15.90:0/959824915 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec01a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebfe00700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 0x7f7ec01a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 -- 172.21.15.90:0/959824915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 msgr2=0x7f7ec01a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 0x7f7ec01a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 -- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 msgr2=0x7f7ec01a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec01a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 -- 172.21.15.90:0/959824915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec01a52b0 con 0x7f7ec010a780 2024-06-02T15:38:15.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.100+0000 7f7ebf400700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec01a0830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0098e0 tx=0x7f7eac005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.101+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac026070 con 0x7f7ec010a780 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.101+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec01a5540 con 0x7f7ec010a780 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.101+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac0056c0 con 0x7f7ec010a780 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.101+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00f960 con 0x7f7ec010a780 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.101+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec01ace20 con 0x7f7ec010a780 2024-06-02T15:38:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.102+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec006a230 con 0x7f7ec010a780 2024-06-02T15:38:15.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.104+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7eac029070 con 0x7f7ec010a780 2024-06-02T15:38:15.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.106+0000 7f7ebcc00700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 0x7f7ea40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.107+0000 7f7ebfe00700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 0x7f7ea40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.107+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f7eac0e2b00 con 0x7f7ec010a780 2024-06-02T15:38:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.107+0000 7f7ebfe00700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 0x7f7ea40b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009c60 tx=0x7f7eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.108+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7eac030020 con 0x7f7ec010a780 2024-06-02T15:38:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.342+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7ec002e560 con 0x7f7ec010a780 2024-06-02T15:38:15.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.435+0000 7f7ebcc00700 1 -- 172.21.15.90:0/959824915 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f7eac013070 con 0x7f7ec010a780 2024-06-02T15:38:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 msgr2=0x7f7ea40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 0x7f7ea40b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009c60 tx=0x7f7eb0009260 comp rx=0 tx=0).stop 2024-06-02T15:38:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 msgr2=0x7f7ec01a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec01a0830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0098e0 tx=0x7f7eac005280 comp rx=0 tx=0).stop 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 shutdown_connections 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ea40b2900 0x7f7ea40b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ec010eeb0 0x7f7ec01a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ec0109dd0 0x7f7ec01a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 --2- 172.21.15.90:0/959824915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ec010a780 0x7f7ec01a0830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:15.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.442+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 >> 172.21.15.90:0/959824915 conn(0x7f7ec0076d10 msgr2=0x7f7ec00788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:15.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.444+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 shutdown_connections 2024-06-02T15:38:15.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:15.444+0000 7f7ec7000700 1 -- 172.21.15.90:0/959824915 wait complete. 2024-06-02T15:38:15.449 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-06-02T15:38:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:15 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/959824915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:15 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:15.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:15 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/959824915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:15.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:15 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:15.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:15 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/959824915' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:15.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:15 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-02T15:38:16.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-02T15:38:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:16 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-02T15:38:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:16 smithi090 ceph-mon[31595]: osdmap e76: 12 total, 12 up, 12 in 2024-06-02T15:38:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:16 smithi090 ceph-mon[31595]: pgmap v315: 97 pgs: 18 active+clean, 9 creating+peering, 70 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:16.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:16 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-02T15:38:16.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:16 smithi120 ceph-mon[36451]: osdmap e76: 12 total, 12 up, 12 in 2024-06-02T15:38:16.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:16 smithi120 ceph-mon[36451]: pgmap v315: 97 pgs: 18 active+clean, 9 creating+peering, 70 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:16.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:16 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-02T15:38:16.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:16 smithi135 ceph-mon[36839]: osdmap e76: 12 total, 12 up, 12 in 2024-06-02T15:38:16.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:16 smithi135 ceph-mon[36839]: pgmap v315: 97 pgs: 18 active+clean, 9 creating+peering, 70 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:16.838 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:17 smithi090 ceph-mon[31595]: osdmap e77: 12 total, 12 up, 12 in 2024-06-02T15:38:17.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:17 smithi120 ceph-mon[36451]: osdmap e77: 12 total, 12 up, 12 in 2024-06-02T15:38:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:17 smithi135 ceph-mon[36839]: osdmap e77: 12 total, 12 up, 12 in 2024-06-02T15:38:18.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:18 smithi090 ceph-mon[31595]: osdmap e78: 12 total, 12 up, 12 in 2024-06-02T15:38:18.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:18 smithi090 ceph-mon[31595]: pgmap v318: 97 pgs: 53 active+clean, 16 creating+peering, 28 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:18 smithi120 ceph-mon[36451]: osdmap e78: 12 total, 12 up, 12 in 2024-06-02T15:38:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:18 smithi120 ceph-mon[36451]: pgmap v318: 97 pgs: 53 active+clean, 16 creating+peering, 28 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:18 smithi135 ceph-mon[36839]: osdmap e78: 12 total, 12 up, 12 in 2024-06-02T15:38:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:18 smithi135 ceph-mon[36839]: pgmap v318: 97 pgs: 53 active+clean, 16 creating+peering, 28 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 -- 172.21.15.90:0/3185528674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b41095b0 msgr2=0x7f87b4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 --2- 172.21.15.90:0/3185528674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b41095b0 0x7f87b4109990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009990 tx=0x7f87a8009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 -- 172.21.15.90:0/3185528674 shutdown_connections 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 --2- 172.21.15.90:0/3185528674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 --2- 172.21.15.90:0/3185528674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b41095b0 0x7f87b4109990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 --2- 172.21.15.90:0/3185528674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b406b220 0x7f87b406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 -- 172.21.15.90:0/3185528674 >> 172.21.15.90:0/3185528674 conn(0x7f87b40793f0 msgr2=0x7f87b4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:19.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 -- 172.21.15.90:0/3185528674 shutdown_connections 2024-06-02T15:38:19.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.081+0000 7f87bb600700 1 -- 172.21.15.90:0/3185528674 wait complete. 2024-06-02T15:38:19.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 Processor -- start 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 -- start start 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b406b220 0x7f87b419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41095b0 0x7f87b41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4113730 con 0x7f87b406bbe0 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41135b0 con 0x7f87b406b220 2024-06-02T15:38:19.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.083+0000 7f87bb600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41138b0 con 0x7f87b41095b0 2024-06-02T15:38:19.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:19.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52454/0 (socket says 172.21.15.90:52454) 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 -- 172.21.15.90:0/1219934623 learned_addr learned my addr 172.21.15.90:0/1219934623 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b8c00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b406b220 0x7f87b419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b9600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41095b0 0x7f87b41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 -- 172.21.15.90:0/1219934623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41095b0 msgr2=0x7f87b41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41095b0 0x7f87b41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 -- 172.21.15.90:0/1219934623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b406b220 msgr2=0x7f87b419be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b406b220 0x7f87b419be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.084+0000 7f87b3e00700 1 -- 172.21.15.90:0/1219934623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b41a0e90 con 0x7f87b406bbe0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87b3e00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9c0 tx=0x7f87a000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a000cbd0 con 0x7f87b406bbe0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a000ee50 con 0x7f87b406bbe0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0005550 con 0x7f87b406bbe0 2024-06-02T15:38:19.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b41a1180 con 0x7f87b406bbe0 2024-06-02T15:38:19.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.085+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a8a00 con 0x7f87b406bbe0 2024-06-02T15:38:19.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.087+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b406a230 con 0x7f87b406bbe0 2024-06-02T15:38:19.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.089+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f87a0004750 con 0x7f87b406bbe0 2024-06-02T15:38:19.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.091+0000 7f87b1600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 0x7f879c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:19.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.091+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f87a00e2540 con 0x7f87b406bbe0 2024-06-02T15:38:19.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.091+0000 7f87b8c00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 0x7f879c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:19.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.092+0000 7f87b8c00700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 0x7f879c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:19.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.094+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87a0054df0 con 0x7f87b406bbe0 2024-06-02T15:38:19.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.376+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f87b419d1e0 con 0x7f87b406bbe0 2024-06-02T15:38:19.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.475+0000 7f87b1600700 1 -- 172.21.15.90:0/1219934623 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f87a00aa370 con 0x7f87b406bbe0 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 msgr2=0x7f879c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 0x7f879c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800b260 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 msgr2=0x7f87b419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9c0 tx=0x7f87a000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 shutdown_connections 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f879c0b2900 0x7f879c0b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41095b0 0x7f87b41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87b406b220 0x7f87b419be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 --2- 172.21.15.90:0/1219934623 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b406bbe0 0x7f87b419c3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 >> 172.21.15.90:0/1219934623 conn(0x7f87b40793f0 msgr2=0x7f87b410e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 shutdown_connections 2024-06-02T15:38:19.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:19.480+0000 7f87bb600700 1 -- 172.21.15.90:0/1219934623 wait complete. 2024-06-02T15:38:19.480 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-06-02T15:38:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:19 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T15:38:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:19 smithi120 ceph-mon[36451]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T15:38:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:19 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T15:38:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:19 smithi135 ceph-mon[36839]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T15:38:19.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:19 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T15:38:19.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:19 smithi090 ceph-mon[31595]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T15:38:20.307 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-02T15:38:20.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-02T15:38:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:20 smithi090 ceph-mon[31595]: osdmap e79: 12 total, 12 up, 12 in 2024-06-02T15:38:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T15:38:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:20 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T15:38:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:20 smithi090 ceph-mon[31595]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T15:38:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:20 smithi090 ceph-mon[31595]: pgmap v320: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:20.749 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:20 smithi120 ceph-mon[36451]: osdmap e79: 12 total, 12 up, 12 in 2024-06-02T15:38:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T15:38:20.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:20 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T15:38:20.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:20 smithi120 ceph-mon[36451]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T15:38:20.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:20 smithi120 ceph-mon[36451]: pgmap v320: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:20 smithi135 ceph-mon[36839]: osdmap e79: 12 total, 12 up, 12 in 2024-06-02T15:38:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1219934623' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T15:38:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:20 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T15:38:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:20 smithi135 ceph-mon[36839]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T15:38:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:20 smithi135 ceph-mon[36839]: pgmap v320: 97 pgs: 73 active+clean, 16 creating+peering, 8 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:21.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:21 smithi090 ceph-mon[31595]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T15:38:21.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:21 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:38:21.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:21 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:38:21.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:21 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:21.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:21 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:21 smithi120 ceph-mon[36451]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T15:38:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:21 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:38:21.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:21 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:38:21.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:21 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:21.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:21 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:21 smithi135 ceph-mon[36839]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T15:38:21.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:21 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:38:21.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:21 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:38:21.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:21 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:21.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:21 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:22.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:22 smithi090 ceph-mon[31595]: pgmap v321: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T15:38:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:22 smithi120 ceph-mon[36451]: pgmap v321: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T15:38:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:22 smithi135 ceph-mon[36839]: pgmap v321: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 -- 172.21.15.90:0/3971885308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10eeb0 msgr2=0x7f103c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 --2- 172.21.15.90:0/3971885308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10eeb0 0x7f103c116a80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1034009f90 tx=0x7f103400b330 comp rx=0 tx=0).stop 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 -- 172.21.15.90:0/3971885308 shutdown_connections 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 --2- 172.21.15.90:0/3971885308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10eeb0 0x7f103c116a80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 --2- 172.21.15.90:0/3971885308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 --2- 172.21.15.90:0/3971885308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10a780 0x7f103c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 -- 172.21.15.90:0/3971885308 >> 172.21.15.90:0/3971885308 conn(0x7f103c076d10 msgr2=0x7f103c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 -- 172.21.15.90:0/3971885308 shutdown_connections 2024-06-02T15:38:22.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.897+0000 7f1042a00700 1 -- 172.21.15.90:0/3971885308 wait complete. 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.898+0000 7f1042a00700 1 Processor -- start 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.898+0000 7f1042a00700 1 -- start start 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 0x7f103c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 0x7f103c1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f103c119d90 con 0x7f103c10eeb0 2024-06-02T15:38:22.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f103c119c10 con 0x7f103c10a780 2024-06-02T15:38:22.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1042a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f103c119f10 con 0x7f103c109dd0 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f103be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f1040a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 0x7f103c1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f103be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:47684/0 (socket says 172.21.15.90:47684) 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f103be00700 1 -- 172.21.15.90:0/1301896541 learned_addr learned my addr 172.21.15.90:0/1301896541 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.899+0000 7f103b400700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 0x7f103c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 -- 172.21.15.90:0/1301896541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 msgr2=0x7f103c1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:22.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 -- 172.21.15.90:0/1301896541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 msgr2=0x7f103c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 0x7f103c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 -- 172.21.15.90:0/1301896541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c1a5200 con 0x7f103c10eeb0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f103be00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f103b400700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 0x7f103c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1040a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 0x7f103c1a4ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f103400b6c0 tx=0x7f103400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034021070 con 0x7f103c10eeb0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1034007d60 con 0x7f103c10eeb0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c1a5490 con 0x7f103c10eeb0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.900+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034005040 con 0x7f103c10eeb0 2024-06-02T15:38:22.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.901+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1acd70 con 0x7f103c10eeb0 2024-06-02T15:38:22.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.902+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c06a230 con 0x7f103c10eeb0 2024-06-02T15:38:22.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.905+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1034024070 con 0x7f103c10eeb0 2024-06-02T15:38:22.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.907+0000 7f1038c00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 0x7f10200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:22.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.907+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f103400f020 con 0x7f103c10eeb0 2024-06-02T15:38:22.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.907+0000 7f103be00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 0x7f10200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:22.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.908+0000 7f103be00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 0x7f10200b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f102c011550 tx=0x7f102c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:22.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:22.910+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f103402b020 con 0x7f103c10eeb0 2024-06-02T15:38:23.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.158+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f103c02e560 con 0x7f103c10eeb0 2024-06-02T15:38:23.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.495+0000 7f1038c00700 1 -- 172.21.15.90:0/1301896541 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f10340ab8f0 con 0x7f103c10eeb0 2024-06-02T15:38:23.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 msgr2=0x7f10200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:23.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 0x7f10200b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f102c011550 tx=0x7f102c009260 comp rx=0 tx=0).stop 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 msgr2=0x7f103c1a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 0x7f103c1a4ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f103400b6c0 tx=0x7f103400cc30 comp rx=0 tx=0).stop 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 shutdown_connections 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10200b2880 0x7f10200b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f103c10eeb0 0x7f103c1a4ac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:23.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f103c109dd0 0x7f103c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:23.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 --2- 172.21.15.90:0/1301896541 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f103c10a780 0x7f103c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:23.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 >> 172.21.15.90:0/1301896541 conn(0x7f103c076d10 msgr2=0x7f103c0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:23.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.499+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 shutdown_connections 2024-06-02T15:38:23.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:23.500+0000 7f1042a00700 1 -- 172.21.15.90:0/1301896541 wait complete. 2024-06-02T15:38:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:23 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-02T15:38:23.557 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:38:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:38:23.306+0000 7f024696e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-02T15:38:23.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:23 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-02T15:38:23.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:23 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-02T15:38:24.359 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-02T15:38:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:24 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-02T15:38:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:24 smithi090 ceph-mon[31595]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T15:38:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:24 smithi090 ceph-mon[31595]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-02T15:38:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:24 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-02T15:38:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:24 smithi120 ceph-mon[36451]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T15:38:24.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:24 smithi120 ceph-mon[36451]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-02T15:38:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:24 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1301896541' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-02T15:38:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:24 smithi135 ceph-mon[36839]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T15:38:24.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:24 smithi135 ceph-mon[36839]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-06-02T15:38:26.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:26 smithi090 ceph-mon[31595]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:27.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 -- 172.21.15.90:0/3519275280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 msgr2=0x7f8d0c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:27.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 --2- 172.21.15.90:0/3519275280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c10bb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009990 tx=0x7f8cfc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 -- 172.21.15.90:0/3519275280 shutdown_connections 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 --2- 172.21.15.90:0/3519275280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c106120 0x7f8d0c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 --2- 172.21.15.90:0/3519275280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c10bb40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 --2- 172.21.15.90:0/3519275280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c105760 0x7f8d0c105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 -- 172.21.15.90:0/3519275280 >> 172.21.15.90:0/3519275280 conn(0x7f8d0c101250 msgr2=0x7f8d0c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 -- 172.21.15.90:0/3519275280 shutdown_connections 2024-06-02T15:38:27.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.042+0000 7f8d11e00700 1 -- 172.21.15.90:0/3519275280 wait complete. 2024-06-02T15:38:27.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.043+0000 7f8d11e00700 1 Processor -- start 2024-06-02T15:38:27.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 -- start start 2024-06-02T15:38:27.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c106120 0x7f8d0c19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c115940 con 0x7f8d0c10b760 2024-06-02T15:38:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1157c0 con 0x7f8d0c105760 2024-06-02T15:38:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d11e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c115ac0 con 0x7f8d0c106120 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d0aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d0a000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c106120 0x7f8d0c19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d0b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d0aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36192/0 (socket says 172.21.15.90:36192) 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.044+0000 7f8d0aa00700 1 -- 172.21.15.90:0/343161427 learned_addr learned my addr 172.21.15.90:0/343161427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 -- 172.21.15.90:0/343161427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c106120 msgr2=0x7f8d0c19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c106120 0x7f8d0c19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 -- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 msgr2=0x7f8d0c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 -- 172.21.15.90:0/343161427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1a0da0 con 0x7f8d0c105760 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.045+0000 7f8d0aa00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc000c00 tx=0x7f8cfc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.046+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc027070 con 0x7f8d0c105760 2024-06-02T15:38:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.046+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c1a1030 con 0x7f8d0c105760 2024-06-02T15:38:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.046+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc022470 con 0x7f8d0c105760 2024-06-02T15:38:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.046+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc017650 con 0x7f8d0c105760 2024-06-02T15:38:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.046+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c1a8910 con 0x7f8d0c105760 2024-06-02T15:38:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.047+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c10dcf0 con 0x7f8d0c105760 2024-06-02T15:38:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.049+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8cfc02a070 con 0x7f8d0c105760 2024-06-02T15:38:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.051+0000 7f8d10a00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 0x7f8cf00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.051+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f8cfc0e3080 con 0x7f8d0c105760 2024-06-02T15:38:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.051+0000 7f8d0a000700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 0x7f8cf00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.052+0000 7f8d0a000700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 0x7f8cf00b4e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c10ab30 tx=0x7f8cf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:27.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.053+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8cfc013070 con 0x7f8d0c105760 2024-06-02T15:38:27.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:26 smithi120 ceph-mon[36451]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:27.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:26 smithi135 ceph-mon[36839]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:27.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:27.286+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8d0c19d0f0 con 0x7f8d0c105760 2024-06-02T15:38:28.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:27 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/343161427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:27 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:27 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/343161427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:27 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:27 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/343161427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:27 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T15:38:28.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.753+0000 7f8d10a00700 1 -- 172.21.15.90:0/343161427 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f8cfc0a4e20 con 0x7f8d0c105760 2024-06-02T15:38:28.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.763+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 msgr2=0x7f8cf00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.763+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 0x7f8cf00b4e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c10ab30 tx=0x7f8cf8009260 comp rx=0 tx=0).stop 2024-06-02T15:38:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.763+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 msgr2=0x7f8d0c19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.763+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc000c00 tx=0x7f8cfc00bfd0 comp rx=0 tx=0).stop 2024-06-02T15:38:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 shutdown_connections 2024-06-02T15:38:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cf00b29b0 0x7f8cf00b4e70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d0c10b760 0x7f8d0c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d0c105760 0x7f8d0c19bd90 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc000c00 tx=0x7f8cfc00bfd0 comp rx=0 tx=0).stop 2024-06-02T15:38:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 --2- 172.21.15.90:0/343161427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c106120 0x7f8d0c19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.764+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 >> 172.21.15.90:0/343161427 conn(0x7f8d0c101250 msgr2=0x7f8d0c1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.766+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 shutdown_connections 2024-06-02T15:38:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:28.766+0000 7f8d11e00700 1 -- 172.21.15.90:0/343161427 wait complete. 2024-06-02T15:38:28.767 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' created 2024-06-02T15:38:29.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:28 smithi090 ceph-mon[31595]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:29.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:28 smithi090 ceph-mon[31595]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T15:38:29.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:28 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T15:38:29.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:28 smithi090 ceph-mon[31595]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T15:38:29.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:28 smithi120 ceph-mon[36451]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:29.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:28 smithi120 ceph-mon[36451]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T15:38:29.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:28 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T15:38:29.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:28 smithi120 ceph-mon[36451]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T15:38:29.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:28 smithi135 ceph-mon[36839]: pgmap v325: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T15:38:29.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:28 smithi135 ceph-mon[36839]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T15:38:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:28 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T15:38:29.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:28 smithi135 ceph-mon[36839]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T15:38:29.602 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-02T15:38:30.058 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:31.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:30 smithi120 ceph-mon[36451]: pgmap v328: 129 pgs: 17 creating+peering, 15 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 1 op/s 2024-06-02T15:38:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:30 smithi120 ceph-mon[36451]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T15:38:31.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:30 smithi120 ceph-mon[36451]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:31.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:30 smithi135 ceph-mon[36839]: pgmap v328: 129 pgs: 17 creating+peering, 15 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 1 op/s 2024-06-02T15:38:31.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:30 smithi135 ceph-mon[36839]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T15:38:31.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:30 smithi135 ceph-mon[36839]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:31.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:30 smithi090 ceph-mon[31595]: pgmap v328: 129 pgs: 17 creating+peering, 15 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s wr, 1 op/s 2024-06-02T15:38:31.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:30 smithi090 ceph-mon[31595]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T15:38:31.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:30 smithi090 ceph-mon[31595]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T15:38:31.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:31 smithi090 ceph-mon[31595]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T15:38:32.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:31 smithi120 ceph-mon[36451]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T15:38:32.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:31 smithi135 ceph-mon[36839]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 -- 172.21.15.90:0/1787831412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 msgr2=0x7f26f81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 --2- 172.21.15.90:0/1787831412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f81028c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009990 tx=0x7f26e4009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 -- 172.21.15.90:0/1787831412 shutdown_connections 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 --2- 172.21.15.90:0/1787831412 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 --2- 172.21.15.90:0/1787831412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 0x7f26f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 --2- 172.21.15.90:0/1787831412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f81028c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 -- 172.21.15.90:0/1787831412 >> 172.21.15.90:0/1787831412 conn(0x7f26f80793f0 msgr2=0x7f26f8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:32.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 -- 172.21.15.90:0/1787831412 shutdown_connections 2024-06-02T15:38:32.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.496+0000 7f26ffc00700 1 -- 172.21.15.90:0/1787831412 wait complete. 2024-06-02T15:38:32.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.497+0000 7f26ffc00700 1 Processor -- start 2024-06-02T15:38:32.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 -- start start 2024-06-02T15:38:32.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f8104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:32.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f81045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:32.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 0x7f26f8104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8112620 con 0x7f26f8102440 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81124a0 con 0x7f26f8102e00 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26ffc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f81127a0 con 0x7f26f8108520 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26fd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f8104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26fd200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f8104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56294/0 (socket says 172.21.15.90:56294) 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26fd200700 1 -- 172.21.15.90:0/2791144061 learned_addr learned my addr 172.21.15.90:0/2791144061 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26f7e00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f81045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.498+0000 7f26fdc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 0x7f26f8104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 -- 172.21.15.90:0/2791144061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 msgr2=0x7f26f8104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 0x7f26f8104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 -- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 msgr2=0x7f26f8104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f8104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 -- 172.21.15.90:0/2791144061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f81a8250 con 0x7f26f8102e00 2024-06-02T15:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f7e00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f81045c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4026070 con 0x7f26f8102e00 2024-06-02T15:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e4021410 con 0x7f26f8102e00 2024-06-02T15:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f81a84e0 con 0x7f26f8102e00 2024-06-02T15:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.499+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4017650 con 0x7f26f8102e00 2024-06-02T15:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.500+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81a8990 con 0x7f26f8102e00 2024-06-02T15:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.501+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f806a230 con 0x7f26f8102e00 2024-06-02T15:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.503+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f26e4029070 con 0x7f26f8102e00 2024-06-02T15:38:32.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.504+0000 7f26f5600700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 0x7f26e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:32.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.504+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f26e40e37f0 con 0x7f26f8102e00 2024-06-02T15:38:32.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.504+0000 7f26fd200700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 0x7f26e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:32.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.505+0000 7f26fd200700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 0x7f26e00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ca0 tx=0x7f26ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:32.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.506+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26e4030070 con 0x7f26f8102e00 2024-06-02T15:38:32.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.734+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f26f802e560 con 0x7f26f8102e00 2024-06-02T15:38:32.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.782+0000 7f26f5600700 1 -- 172.21.15.90:0/2791144061 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f26e4013070 con 0x7f26f8102e00 2024-06-02T15:38:32.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 msgr2=0x7f26e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:32.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 0x7f26e00b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ca0 tx=0x7f26ec009260 comp rx=0 tx=0).stop 2024-06-02T15:38:32.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 msgr2=0x7f26f81045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f81045c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40047b0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 shutdown_connections 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e00b2900 0x7f26e00b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f8108520 0x7f26f8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26f8102440 0x7f26f8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.786+0000 7f26ffc00700 1 --2- 172.21.15.90:0/2791144061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26f8102e00 0x7f26f81045c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.787+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 >> 172.21.15.90:0/2791144061 conn(0x7f26f80793f0 msgr2=0x7f26f806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.787+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 shutdown_connections 2024-06-02T15:38:32.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:32.787+0000 7f26ffc00700 1 -- 172.21.15.90:0/2791144061 wait complete. 2024-06-02T15:38:32.786 INFO:teuthology.orchestra.run.smithi090.stderr:set pool 4 allow_ec_overwrites to true 2024-06-02T15:38:33.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:32 smithi120 ceph-mon[36451]: pgmap v331: 129 pgs: 17 creating+peering, 4 unknown, 108 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:33.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:32 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2791144061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:32 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:32 smithi135 ceph-mon[36839]: pgmap v331: 129 pgs: 17 creating+peering, 4 unknown, 108 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:33.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:32 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2791144061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:32 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:32 smithi090 ceph-mon[31595]: pgmap v331: 129 pgs: 17 creating+peering, 4 unknown, 108 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:32 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2791144061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:32 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-02T15:38:33.579 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-02T15:38:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:34.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:33 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-02T15:38:34.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:33 smithi120 ceph-mon[36451]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T15:38:34.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:33 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-02T15:38:34.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:33 smithi135 ceph-mon[36839]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T15:38:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:33 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-02T15:38:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:33 smithi090 ceph-mon[31595]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T15:38:35.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:34 smithi120 ceph-mon[36451]: pgmap v333: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:35.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:34 smithi135 ceph-mon[36839]: pgmap v333: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:35.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:34 smithi090 ceph-mon[31595]: pgmap v333: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 -- 172.21.15.90:0/2642132216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250106110 msgr2=0x7f6250112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 --2- 172.21.15.90:0/2642132216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250106110 0x7f6250112620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6248009f90 tx=0x7f624800b330 comp rx=0 tx=0).stop 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 -- 172.21.15.90:0/2642132216 shutdown_connections 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 --2- 172.21.15.90:0/2642132216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250106110 0x7f6250112620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 --2- 172.21.15.90:0/2642132216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f625010b750 0x7f625010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 --2- 172.21.15.90:0/2642132216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6250105750 0x7f6250105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 -- 172.21.15.90:0/2642132216 >> 172.21.15.90:0/2642132216 conn(0x7f6250101220 msgr2=0x7f6250103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 -- 172.21.15.90:0/2642132216 shutdown_connections 2024-06-02T15:38:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.226+0000 7f6255400700 1 -- 172.21.15.90:0/2642132216 wait complete. 2024-06-02T15:38:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.227+0000 7f6255400700 1 Processor -- start 2024-06-02T15:38:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.227+0000 7f6255400700 1 -- start start 2024-06-02T15:38:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 0x7f625019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 0x7f625019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 0x7f62501a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6250115980 con 0x7f6250106110 2024-06-02T15:38:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6250115800 con 0x7f6250105750 2024-06-02T15:38:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f6255400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6250115b00 con 0x7f625010b750 2024-06-02T15:38:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f624de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 0x7f625019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f624f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 0x7f62501a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f624de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 0x7f625019c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56316/0 (socket says 172.21.15.90:56316) 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f624de00700 1 -- 172.21.15.90:0/1806777814 learned_addr learned my addr 172.21.15.90:0/1806777814 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.228+0000 7f624e800700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 0x7f625019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 -- 172.21.15.90:0/1806777814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 msgr2=0x7f625019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 0x7f625019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 -- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 msgr2=0x7f625019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 0x7f625019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 -- 172.21.15.90:0/1806777814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62501a0dc0 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624e800700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 0x7f625019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f624f200700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 0x7f62501a0680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f624800b6c0 tx=0x7f624800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248021070 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6248007ac0 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62501a1050 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248005040 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.229+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62501a8930 con 0x7f625010b750 2024-06-02T15:38:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.231+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625006a230 con 0x7f625010b750 2024-06-02T15:38:36.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.233+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6248024070 con 0x7f625010b750 2024-06-02T15:38:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.234+0000 7f6247400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 0x7f62340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.234+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f624800f020 con 0x7f625010b750 2024-06-02T15:38:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.234+0000 7f624e800700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 0x7f62340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.235+0000 7f624e800700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 0x7f62340b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6240005fd0 tx=0x7f6240009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:36.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.236+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f624802b020 con 0x7f625010b750 2024-06-02T15:38:36.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.524+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f625002e560 con 0x7f625010b750 2024-06-02T15:38:36.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.816+0000 7f6247400700 1 -- 172.21.15.90:0/1806777814 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f62480ab590 con 0x7f625010b750 2024-06-02T15:38:36.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.822+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 msgr2=0x7f62340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:36.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.822+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 0x7f62340b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6240005fd0 tx=0x7f6240009310 comp rx=0 tx=0).stop 2024-06-02T15:38:36.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 msgr2=0x7f62501a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:36.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 0x7f62501a0680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f624800b6c0 tx=0x7f624800cc30 comp rx=0 tx=0).stop 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 shutdown_connections 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62340b2880 0x7f62340b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f625010b750 0x7f62501a0680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6250105750 0x7f625019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 --2- 172.21.15.90:0/1806777814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6250106110 0x7f625019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 >> 172.21.15.90:0/1806777814 conn(0x7f6250101220 msgr2=0x7f62501017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 shutdown_connections 2024-06-02T15:38:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:36.823+0000 7f6255400700 1 -- 172.21.15.90:0/1806777814 wait complete. 2024-06-02T15:38:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:added data pool 4 to fsmap 2024-06-02T15:38:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:36 smithi090 ceph-mon[31595]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:36 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1806777814' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:36 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:36 smithi120 ceph-mon[36451]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:36 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1806777814' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:36 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:36 smithi135 ceph-mon[36839]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:36 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1806777814' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:36 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-02T15:38:37.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:38:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:37 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-02T15:38:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:37 smithi090 ceph-mon[31595]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T15:38:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:37 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:38.192 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:38.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:37 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-02T15:38:38.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:37 smithi120 ceph-mon[36451]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T15:38:38.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:37 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:38.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:37 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-02T15:38:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:37 smithi135 ceph-mon[36839]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T15:38:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:37 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:38:38.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:38 smithi090 ceph-mon[31595]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:38.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:38 smithi090 ceph-mon[31595]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:38:38.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:38 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:38:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:38 smithi120 ceph-mon[36451]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:38 smithi120 ceph-mon[36451]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:38:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:38 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:38:39.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:38 smithi135 ceph-mon[36839]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:39.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:38 smithi135 ceph-mon[36839]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T15:38:39.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:38 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:38:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 -- 172.21.15.90:0/2654637540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e80106120 msgr2=0x7f6e80112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 --2- 172.21.15.90:0/2654637540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e80106120 0x7f6e80112630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800b0a0 tx=0x7f6e7800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 -- 172.21.15.90:0/2654637540 shutdown_connections 2024-06-02T15:38:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 --2- 172.21.15.90:0/2654637540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e80106120 0x7f6e80112630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 --2- 172.21.15.90:0/2654637540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e8010b760 0x7f6e8010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 --2- 172.21.15.90:0/2654637540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80105760 0x7f6e80105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 -- 172.21.15.90:0/2654637540 >> 172.21.15.90:0/2654637540 conn(0x7f6e80101250 msgr2=0x7f6e80103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 -- 172.21.15.90:0/2654637540 shutdown_connections 2024-06-02T15:38:40.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.456+0000 7f6e85c00700 1 -- 172.21.15.90:0/2654637540 wait complete. 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.457+0000 7f6e85c00700 1 Processor -- start 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 -- start start 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 0x7f6e8019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 0x7f6e8019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e8010b760 0x7f6e801a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6e80115960 con 0x7f6e80105760 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6e801157e0 con 0x7f6e80106120 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e85c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e80115ae0 con 0x7f6e8010b760 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e7ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 0x7f6e8019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e7e000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 0x7f6e8019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:40.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e7ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 0x7f6e8019be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56326/0 (socket says 172.21.15.90:56326) 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e7ea00700 1 -- 172.21.15.90:0/882770108 learned_addr learned my addr 172.21.15.90:0/882770108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.458+0000 7f6e7f400700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e8010b760 0x7f6e801a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 -- 172.21.15.90:0/882770108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e8010b760 msgr2=0x7f6e801a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e8010b760 0x7f6e801a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 -- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 msgr2=0x7f6e8019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 0x7f6e8019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 -- 172.21.15.90:0/882770108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e801a0e40 con 0x7f6e80106120 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e7e000700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 0x7f6e8019c370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e9f0 tx=0x7f6e6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c004220 con 0x7f6e80106120 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e6c0043c0 con 0x7f6e80106120 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e801a1130 con 0x7f6e80106120 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c005020 con 0x7f6e80106120 2024-06-02T15:38:40.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.459+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801a89b0 con 0x7f6e80106120 2024-06-02T15:38:40.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.460+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8006a230 con 0x7f6e80106120 2024-06-02T15:38:40.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.464+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6e6c004750 con 0x7f6e80106120 2024-06-02T15:38:40.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.465+0000 7f6e77400700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 0x7f6e640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:40.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.465+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f6e6c0e24c0 con 0x7f6e80106120 2024-06-02T15:38:40.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.466+0000 7f6e7ea00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 0x7f6e640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:40.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.466+0000 7f6e7ea00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 0x7f6e640b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e70007c10 tx=0x7f6e70007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:40.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.467+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e6c0a9fd0 con 0x7f6e80106120 2024-06-02T15:38:40.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.753+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e8019d1c0 con 0x7f6e80106120 2024-06-02T15:38:40.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.755+0000 7f6e77400700 1 -- 172.21.15.90:0/882770108 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7f6e8019d1c0 con 0x7f6e80106120 2024-06-02T15:38:40.752 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:40.753 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:38:36.527426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-06-02T15:38:40.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 msgr2=0x7f6e640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:40.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 0x7f6e640b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6e70007c10 tx=0x7f6e70007480 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 msgr2=0x7f6e8019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 0x7f6e8019c370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e9f0 tx=0x7f6e6c00ed00 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 shutdown_connections 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6e640b2880 0x7f6e640b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e8010b760 0x7f6e801a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e80105760 0x7f6e8019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 --2- 172.21.15.90:0/882770108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6e80106120 0x7f6e8019c370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:40.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 >> 172.21.15.90:0/882770108 conn(0x7f6e80101250 msgr2=0x7f6e801017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.758+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 shutdown_connections 2024-06-02T15:38:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:40.759+0000 7f6e85c00700 1 -- 172.21.15.90:0/882770108 wait complete. 2024-06-02T15:38:40.757 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-06-02T15:38:41.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:40 smithi135 ceph-mon[36839]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:41.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/882770108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:41.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:40 smithi090 ceph-mon[31595]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:41.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/882770108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:40 smithi120 ceph-mon[36451]: pgmap v337: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/882770108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:41.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:38:42.000 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:43.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:42 smithi090 ceph-mon[31595]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:43.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:42 smithi120 ceph-mon[36451]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:43.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:42 smithi135 ceph-mon[36839]: pgmap v338: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 -- 172.21.15.90:0/233223281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 msgr2=0x7fd48010ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 --2- 172.21.15.90:0/233223281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd48010ff10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd47800b0a0 tx=0x7fd47800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 -- 172.21.15.90:0/233223281 shutdown_connections 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 --2- 172.21.15.90:0/233223281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd48010ff10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 --2- 172.21.15.90:0/233223281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd480104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 --2- 172.21.15.90:0/233223281 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 0x7fd48010d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 -- 172.21.15.90:0/233223281 >> 172.21.15.90:0/233223281 conn(0x7fd4800fde60 msgr2=0x7fd480100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 -- 172.21.15.90:0/233223281 shutdown_connections 2024-06-02T15:38:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.202+0000 7fd47f200700 1 -- 172.21.15.90:0/233223281 wait complete. 2024-06-02T15:38:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.203+0000 7fd47f200700 1 Processor -- start 2024-06-02T15:38:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.203+0000 7fd47f200700 1 -- start start 2024-06-02T15:38:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.203+0000 7fd47f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd48019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.203+0000 7fd47f200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 0x7fd48019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801136a0 con 0x7fd48010da70 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47f200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113520 con 0x7fd480104fb0 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd480113820 con 0x7fd480104690 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37168/0 (socket says 172.21.15.90:37168) 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd48019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 -- 172.21.15.90:0/1728538070 learned_addr learned my addr 172.21.15.90:0/1728538070 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:44.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47d400700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 0x7fd48019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 -- 172.21.15.90:0/1728538070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 msgr2=0x7fd48019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd48019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 -- 172.21.15.90:0/1728538070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 msgr2=0x7fd48019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 0x7fd48019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47e800700 1 -- 172.21.15.90:0/1728538070 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019a120 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.204+0000 7fd47de00700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd48019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd47e800700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd47800b070 tx=0x7fd478009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478021070 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019a3b0 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478004e40 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004630 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.205+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a6680 con 0x7fd48010da70 2024-06-02T15:38:44.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.206+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48006a230 con 0x7fd48010da70 2024-06-02T15:38:44.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.210+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd478007980 con 0x7fd48010da70 2024-06-02T15:38:44.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.212+0000 7fd476a00700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 0x7fd4640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:44.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.212+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd4780e3280 con 0x7fd48010da70 2024-06-02T15:38:44.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.212+0000 7fd47de00700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 0x7fd4640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:44.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.213+0000 7fd47de00700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 0x7fd4640b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd480101940 tx=0x7fd470009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:44.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.215+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4780aad90 con 0x7fd48010da70 2024-06-02T15:38:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.454+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd48019aea0 con 0x7fd48010da70 2024-06-02T15:38:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.457+0000 7fd476a00700 1 -- 172.21.15.90:0/1728538070 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24140 (secure 0 0 0) 0x7fd4780aa690 con 0x7fd48010da70 2024-06-02T15:38:44.455 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:44.455 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:38:36.527196+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.460+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 msgr2=0x7fd4640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.460+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 0x7fd4640b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd480101940 tx=0x7fd470009260 comp rx=0 tx=0).stop 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.460+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 msgr2=0x7fd4801999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.460+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd47800b070 tx=0x7fd478009c90 comp rx=0 tx=0).stop 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 shutdown_connections 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd4640b2880 0x7fd4640b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd48010da70 0x7fd4801999c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd480104690 0x7fd48019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 --2- 172.21.15.90:0/1728538070 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd480104fb0 0x7fd48019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 >> 172.21.15.90:0/1728538070 conn(0x7fd4800fde60 msgr2=0x7fd4800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 shutdown_connections 2024-06-02T15:38:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:44.461+0000 7fd47f200700 1 -- 172.21.15.90:0/1728538070 wait complete. 2024-06-02T15:38:45.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:44 smithi090 ceph-mon[31595]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:45.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1728538070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:44 smithi120 ceph-mon[36451]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:44 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1728538070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:45.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:44 smithi135 ceph-mon[36839]: pgmap v339: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:45.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:44 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1728538070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:45.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:38:45.772 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:47.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:46 smithi090 ceph-mon[31595]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:47.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:46 smithi120 ceph-mon[36451]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:46 smithi135 ceph-mon[36839]: pgmap v340: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 -- 172.21.15.90:0/2757202167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 msgr2=0x7f2cdc10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/2757202167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc10fa00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098f0 tx=0x7f2cc4009c00 comp rx=0 tx=0).stop 2024-06-02T15:38:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 -- 172.21.15.90:0/2757202167 shutdown_connections 2024-06-02T15:38:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/2757202167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 0x7f2cdc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/2757202167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/2757202167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc10fa00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 -- 172.21.15.90:0/2757202167 >> 172.21.15.90:0/2757202167 conn(0x7f2cdc07a6a0 msgr2=0x7f2cdc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 -- 172.21.15.90:0/2757202167 shutdown_connections 2024-06-02T15:38:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.051+0000 7f2ce0c00700 1 -- 172.21.15.90:0/2757202167 wait complete. 2024-06-02T15:38:48.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.052+0000 7f2ce0c00700 1 Processor -- start 2024-06-02T15:38:48.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 -- start start 2024-06-02T15:38:48.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:48.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 0x7f2cdc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc115910 con 0x7f2cdc10ff40 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc115790 con 0x7f2cdc076090 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2ce0c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc115a90 con 0x7f2cdc075770 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2cda800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 0x7f2cdc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2cd9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:49326/0 (socket says 172.21.15.90:49326) 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2cd9e00700 1 -- 172.21.15.90:0/1043812015 learned_addr learned my addr 172.21.15.90:0/1043812015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.053+0000 7f2cd1400700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 -- 172.21.15.90:0/1043812015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 msgr2=0x7f2cdc19c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc19c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 -- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 msgr2=0x7f2cdc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 0x7f2cdc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 -- 172.21.15.90:0/1043812015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc1a0ed0 con 0x7f2cdc075770 2024-06-02T15:38:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd9e00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00e9c0 tx=0x7f2ccc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00cbd0 con 0x7f2cdc075770 2024-06-02T15:38:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc00ee60 con 0x7f2cdc075770 2024-06-02T15:38:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.054+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0055c0 con 0x7f2cdc075770 2024-06-02T15:38:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.055+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc1a11c0 con 0x7f2cdc075770 2024-06-02T15:38:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.055+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc1a8a40 con 0x7f2cdc075770 2024-06-02T15:38:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.056+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc06a230 con 0x7f2cdc075770 2024-06-02T15:38:48.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.058+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2ccc004750 con 0x7f2cdc075770 2024-06-02T15:38:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.059+0000 7f2cd3400700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 0x7f2cc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.060+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f2ccc0e2730 con 0x7f2cdc075770 2024-06-02T15:38:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.060+0000 7f2cd1400700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 0x7f2cc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.060+0000 7f2cd1400700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 0x7f2cc00b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009810 tx=0x7f2cc4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.063+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ccc0aa1b0 con 0x7f2cdc075770 2024-06-02T15:38:48.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.309+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cdc19d220 con 0x7f2cdc075770 2024-06-02T15:38:48.309 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:48.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.311+0000 7f2cd3400700 1 -- 172.21.15.90:0/1043812015 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24140 (secure 0 0 0) 0x7f2ccc0a9ab0 con 0x7f2cdc075770 2024-06-02T15:38:48.310 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:38:36.527196+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 msgr2=0x7f2cc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 0x7f2cc00b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009810 tx=0x7f2cc4019040 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 msgr2=0x7f2cdc19bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00e9c0 tx=0x7f2ccc00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 shutdown_connections 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cc00b2880 0x7f2cc00b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cdc10ff40 0x7f2cdc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.314+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cdc075770 0x7f2cdc19bec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.315+0000 7f2ce0c00700 1 --2- 172.21.15.90:0/1043812015 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cdc076090 0x7f2cdc19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:48.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.315+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 >> 172.21.15.90:0/1043812015 conn(0x7f2cdc07a6a0 msgr2=0x7f2cdc1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:48.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.315+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 shutdown_connections 2024-06-02T15:38:48.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:48.315+0000 7f2ce0c00700 1 -- 172.21.15.90:0/1043812015 wait complete. 2024-06-02T15:38:49.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:48 smithi090 ceph-mon[31595]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:49.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:48 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1043812015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:49.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:38:49.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:48 smithi120 ceph-mon[36451]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:49.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:48 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1043812015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:49.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:48 smithi135 ceph-mon[36839]: pgmap v341: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:49.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:48 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1043812015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:49.644 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:50.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:50 smithi090 ceph-mon[31595]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:51.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:50 smithi120 ceph-mon[36451]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:51.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:50 smithi135 ceph-mon[36839]: pgmap v342: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 -- 172.21.15.90:0/1751514995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001041a0 msgr2=0x7f19001081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 --2- 172.21.15.90:0/1751514995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001041a0 0x7f19001081c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009990 tx=0x7f18ec009ca0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 -- 172.21.15.90:0/1751514995 shutdown_connections 2024-06-02T15:38:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 --2- 172.21.15.90:0/1751514995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001088f0 0x7f1900110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 --2- 172.21.15.90:0/1751514995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 --2- 172.21.15.90:0/1751514995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001041a0 0x7f19001081c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 -- 172.21.15.90:0/1751514995 >> 172.21.15.90:0/1751514995 conn(0x7f190007a6a0 msgr2=0x7f190007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 -- 172.21.15.90:0/1751514995 shutdown_connections 2024-06-02T15:38:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.864+0000 7f1908200700 1 -- 172.21.15.90:0/1751514995 wait complete. 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.865+0000 7f1908200700 1 Processor -- start 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 -- start start 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001088f0 0x7f19000762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1900113700 con 0x7f19001041a0 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1900113580 con 0x7f19001037f0 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1908200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1900113880 con 0x7f19001088f0 2024-06-02T15:38:51.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1904e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:51.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1905800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:51.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1904e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38024/0 (socket says 172.21.15.90:38024) 2024-06-02T15:38:51.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.866+0000 7f1904e00700 1 -- 172.21.15.90:0/1321124150 learned_addr learned my addr 172.21.15.90:0/1321124150 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:51.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1906200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001088f0 0x7f19000762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 -- 172.21.15.90:0/1321124150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001088f0 msgr2=0x7f19000762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001088f0 0x7f19000762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 -- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 msgr2=0x7f1900075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 -- 172.21.15.90:0/1321124150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19001a80e0 con 0x7f19001037f0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1904e00700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.867+0000 7f1905800700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900075840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.868+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400c9c0 con 0x7f19001037f0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.868+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19001a83d0 con 0x7f19001037f0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.868+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f400cb60 con 0x7f19001037f0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.868+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4018680 con 0x7f19001037f0 2024-06-02T15:38:51.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.868+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19001a8960 con 0x7f19001037f0 2024-06-02T15:38:51.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.869+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190006a230 con 0x7f19001037f0 2024-06-02T15:38:51.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.871+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f18f4010430 con 0x7f19001037f0 2024-06-02T15:38:51.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.873+0000 7f18fe000700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 0x7f18e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:51.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.873+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f18f40e2980 con 0x7f19001037f0 2024-06-02T15:38:51.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.873+0000 7f1904e00700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 0x7f18e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.874+0000 7f1904e00700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 0x7f18e80b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b790 tx=0x7f18ec00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:51.875+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18f40aa400 con 0x7f19001037f0 2024-06-02T15:38:52.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.158+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f190002e560 con 0x7f19001037f0 2024-06-02T15:38:52.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.159+0000 7f18fe000700 1 -- 172.21.15.90:0/1321124150 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7f18f40a9d00 con 0x7f19001037f0 2024-06-02T15:38:52.157 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:52.157 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24512,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:38:36.527426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24536},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24536,"qdb_cluster":[24536]},"id":1}]} 2024-06-02T15:38:52.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 msgr2=0x7f18e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:52.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 0x7f18e80b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b790 tx=0x7f18ec00b280 comp rx=0 tx=0).stop 2024-06-02T15:38:52.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 msgr2=0x7f1900075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900075840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400ed00 comp rx=0 tx=0).stop 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 shutdown_connections 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18e80b2880 0x7f18e80b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f19001088f0 0x7f19000762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f19001037f0 0x7f1900075840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 --2- 172.21.15.90:0/1321124150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19001041a0 0x7f1900075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:52.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 >> 172.21.15.90:0/1321124150 conn(0x7f190007a6a0 msgr2=0x7f19001018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:52.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 shutdown_connections 2024-06-02T15:38:52.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:52.162+0000 7f1908200700 1 -- 172.21.15.90:0/1321124150 wait complete. 2024-06-02T15:38:52.161 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-06-02T15:38:52.984 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:38:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:53 smithi090 ceph-mon[31595]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:53 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1321124150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:53 smithi120 ceph-mon[36451]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:53 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1321124150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:53 smithi135 ceph-mon[36839]: pgmap v343: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:53 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1321124150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:38:55.468 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:55 smithi090 ceph-mon[31595]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:55 smithi120 ceph-mon[36451]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:55 smithi135 ceph-mon[36839]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 -- 172.21.15.90:0/2034365336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 msgr2=0x7f06cc116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 --2- 172.21.15.90:0/2034365336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc116a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06c40099c0 tx=0x7f06c4009cd0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 -- 172.21.15.90:0/2034365336 shutdown_connections 2024-06-02T15:38:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 --2- 172.21.15.90:0/2034365336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc116a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 --2- 172.21.15.90:0/2034365336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc109dd0 0x7f06cc10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 --2- 172.21.15.90:0/2034365336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc10a780 0x7f06cc10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 -- 172.21.15.90:0/2034365336 >> 172.21.15.90:0/2034365336 conn(0x7f06cc076d10 msgr2=0x7f06cc108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.637+0000 7f06d4200700 1 -- 172.21.15.90:0/2034365336 shutdown_connections 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.638+0000 7f06d4200700 1 -- 172.21.15.90:0/2034365336 wait complete. 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.639+0000 7f06d4200700 1 Processor -- start 2024-06-02T15:38:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.639+0000 7f06d4200700 1 -- start start 2024-06-02T15:38:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.639+0000 7f06d4200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 0x7f06cc1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d4200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc10a780 0x7f06cc1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d4200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d4200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119d60 con 0x7f06cc10eeb0 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d4200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119be0 con 0x7f06cc109dd0 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d4200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc119ee0 con 0x7f06cc10a780 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d2200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 0x7f06cc1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d2200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38044/0 (socket says 172.21.15.90:38044) 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d2200700 1 -- 172.21.15.90:0/1050244246 learned_addr learned my addr 172.21.15.90:0/1050244246 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d0e00700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc10a780 0x7f06cc1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 -- 172.21.15.90:0/1050244246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc10a780 msgr2=0x7f06cc1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc10a780 0x7f06cc1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 -- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 msgr2=0x7f06cc1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d1800700 1 -- 172.21.15.90:0/1050244246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc1a51a0 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.640+0000 7f06d2200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06d1800700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 0x7f06cc1a0190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06c000efc0 tx=0x7f06c000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c00092e0 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc1a5430 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c0004bb0 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0004040 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.641+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc1acd10 con 0x7f06cc109dd0 2024-06-02T15:38:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.642+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc06a230 con 0x7f06cc109dd0 2024-06-02T15:38:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.644+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f06c001b030 con 0x7f06cc109dd0 2024-06-02T15:38:55.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.646+0000 7f06ca000700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:55.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.646+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f06c00e2a10 con 0x7f06cc109dd0 2024-06-02T15:38:55.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.646+0000 7f06d0e00700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:55.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.647+0000 7f06d0e00700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 0x7f06b40b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009680 tx=0x7f06b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:55.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.648+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06c0022020 con 0x7f06cc109dd0 2024-06-02T15:38:55.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.880+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06cc02e560 con 0x7f06cc109dd0 2024-06-02T15:38:55.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.881+0000 7f06ca000700 1 -- 172.21.15.90:0/1050244246 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24140 (secure 0 0 0) 0x7f06c0010040 con 0x7f06cc109dd0 2024-06-02T15:38:55.879 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:38:55.880 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:38:36.527196+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:38:55.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.884+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 msgr2=0x7f06b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:55.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.884+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 0x7f06b40b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009680 tx=0x7f06b8009260 comp rx=0 tx=0).stop 2024-06-02T15:38:55.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.884+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 msgr2=0x7f06cc1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:55.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.884+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 0x7f06cc1a0190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06c000efc0 tx=0x7f06c000c3a0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 shutdown_connections 2024-06-02T15:38:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f06b40b2880 0x7f06b40b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06cc10eeb0 0x7f06cc1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f06cc109dd0 0x7f06cc1a0190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 --2- 172.21.15.90:0/1050244246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06cc10a780 0x7f06cc1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:55.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 >> 172.21.15.90:0/1050244246 conn(0x7f06cc076d10 msgr2=0x7f06cc077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:55.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 shutdown_connections 2024-06-02T15:38:55.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:55.885+0000 7f06d4200700 1 -- 172.21.15.90:0/1050244246 wait complete. 2024-06-02T15:38:56.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:56 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1050244246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:56.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:56 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1050244246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:56.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:56 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1050244246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:38:56.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-02T15:38:19.378346+0000', 'modified': '2024-06-02T15:38:36.527426+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24536}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.120:6839/3642613472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 3642613472}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 3642613472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24536, 'qdb_cluster': [24536]} 2024-06-02T15:38:56.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-02T15:38:56.765 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T15:38:57.203 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:38:57.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:57 smithi090 ceph-mon[31595]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:57 smithi120 ceph-mon[36451]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:57 smithi135 ceph-mon[36839]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:58.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-02T15:38:58.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-06-02T15:38:58.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:58 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-02T15:38:58.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-06-02T15:38:58.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:58 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-06-02T15:38:58.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:58 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 -- 172.21.15.90:0/118683976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 msgr2=0x7fd3ec1123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 --2- 172.21.15.90:0/118683976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1123e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b0a0 tx=0x7fd3e400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 -- 172.21.15.90:0/118683976 shutdown_connections 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 --2- 172.21.15.90:0/118683976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1123e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 --2- 172.21.15.90:0/118683976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06b250 0x7fd3ec06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 --2- 172.21.15.90:0/118683976 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 -- 172.21.15.90:0/118683976 >> 172.21.15.90:0/118683976 conn(0x7fd3ec0793f0 msgr2=0x7fd3ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 -- 172.21.15.90:0/118683976 shutdown_connections 2024-06-02T15:38:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.355+0000 7fd3f2000700 1 -- 172.21.15.90:0/118683976 wait complete. 2024-06-02T15:38:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.356+0000 7fd3f2000700 1 Processor -- start 2024-06-02T15:38:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 -- start start 2024-06-02T15:38:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 0x7fd3ec19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec115960 con 0x7fd3ec10fff0 2024-06-02T15:38:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1157e0 con 0x7fd3ec06b250 2024-06-02T15:38:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.357+0000 7fd3f2000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec115ae0 con 0x7fd3ec06bb70 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 0x7fd3ec19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 0x7fd3ec19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:41826/0 (socket says 172.21.15.90:41826) 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eb400700 1 -- 172.21.15.90:0/1043344110 learned_addr learned my addr 172.21.15.90:0/1043344110 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1043344110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 msgr2=0x7fd3ec19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 0x7fd3ec19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 msgr2=0x7fd3ec1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1043344110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec1a0ea0 con 0x7fd3ec06bb70 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.358+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec19c3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e9f0 tx=0x7fd3d800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.359+0000 7fd3e3e00700 1 -- 172.21.15.90:0/1043344110 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800cb70 con 0x7fd3ec06bb70 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.359+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec1a1190 con 0x7fd3ec06bb70 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.359+0000 7fd3e3e00700 1 -- 172.21.15.90:0/1043344110 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d800ee90 con 0x7fd3ec06bb70 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.359+0000 7fd3e3e00700 1 -- 172.21.15.90:0/1043344110 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8003d40 con 0x7fd3ec06bb70 2024-06-02T15:38:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.359+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec1a8a10 con 0x7fd3ec06bb70 2024-06-02T15:38:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.365+0000 7fd3e3e00700 1 -- 172.21.15.90:0/1043344110 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd3d8003ee0 con 0x7fd3ec06bb70 2024-06-02T15:38:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.365+0000 7fd3fc922b80 1 Processor -- start 2024-06-02T15:38:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.365+0000 7fd3fc922b80 1 -- start start 2024-06-02T15:38:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f298559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2984fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558f297addf0 con 0x558f298500f0 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f2979d990 con 0x558f29855ee0 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f297a19d0 con 0x558f298555c0 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38070/0 (socket says 172.21.15.90:38070) 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 -- 172.21.15.90:0/3932781105 learned_addr learned my addr 172.21.15.90:0/3932781105 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f298559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eb400700 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2984fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 -- 172.21.15.90:0/3932781105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 msgr2=0x558f298559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f298559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.366+0000 7fd3eaa00700 1 -- 172.21.15.90:0/3932781105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 msgr2=0x558f2984fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2984fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3eaa00700 1 -- 172.21.15.90:0/3932781105 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f297663d0 con 0x558f298500f0 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80e8a80 tx=0x7fd3d80e8d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4cc4cd6ce8d937 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3e3e00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 0x7fd3d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3e3e00700 1 -- 172.21.15.90:0/1043344110 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd3d80e2600 con 0x7fd3ec06bb70 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 0x7fd3d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.367+0000 7fd3e0c00700 1 -- 172.21.15.90:0/3932781105 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80f1070 con 0x558f298500f0 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.368+0000 7fd3e0c00700 1 -- 172.21.15.90:0/3932781105 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d80a93d0 con 0x558f298500f0 2024-06-02T15:38:59.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.368+0000 7fd3e0c00700 1 -- 172.21.15.90:0/3932781105 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80a9030 con 0x558f298500f0 2024-06-02T15:38:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.368+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3932781105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 msgr2=0x558f29850570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.368+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 0x7fd3d00b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e4006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.368+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80e8a80 tx=0x7fd3d80e8d90 comp rx=0 tx=0).stop 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3932781105 shutdown_connections 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f29850570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f298559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3932781105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2984fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3932781105 >> 172.21.15.90:0/3932781105 conn(0x558f2977db90 msgr2=0x558f2984a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3932781105 shutdown_connections 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3932781105 wait complete. 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.369+0000 7fd3fc922b80 1 -- start start 2024-06-02T15:38:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 Processor -- start 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2985bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 0x558f29859000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f29855ee0 0x558f29859540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558f296a3f50 con 0x558f298500f0 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f2973e710 con 0x558f298555c0 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f2985c990 con 0x558f29855ee0 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3fc922b80 10 client.0 _ll_register_callbacks cb 0x558f29747c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2985bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f29855ee0 0x558f29859540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2985bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38074/0 (socket says 172.21.15.90:38074) 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.370+0000 7fd3eb400700 1 -- 172.21.15.90:0/4173940521 learned_addr learned my addr 172.21.15.90:0/4173940521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 0x558f29859000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 -- 172.21.15.90:0/4173940521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f29855ee0 msgr2=0x558f29859540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f29855ee0 0x558f29859540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 -- 172.21.15.90:0/4173940521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 msgr2=0x558f2985bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2985bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 -- 172.21.15.90:0/4173940521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f296ed880 con 0x558f298555c0 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 0x558f29859000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80a5940 tx=0x7fd3d80e8790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a4df8a3e104eb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3d5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:38:59.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80f1070 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd3d80e2340 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.371+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3d80a93d0 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd3d80a9030 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 10 client.24632 fetch_fsmap learned FSMap version 17 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x558f2976e010 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d80a93d0 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd3d80a5b50 con 0x558f298555c0 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 10 client.24632 fetch_fsmap finished waiting for FSMap version 17 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 10 client.24632 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 20 client.24632 populate_metadata read hostname 'smithi090' 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f29868ca0 0x558f2986a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 4 client.24632 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.372+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f297afd40 con 0x558f29868ca0 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.373+0000 7fd3eb400700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f29868ca0 0x558f2986a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.373+0000 7fd3eb400700 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f29868ca0 0x558f2986a110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d585f3e744945ca server_cookie=ef35e12337cbc49f in_seq=0 out_seq=0 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.373+0000 7fd3d5600700 10 client.24632 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.374+0000 7fd3d5600700 1 -- 172.21.15.90:0/4173940521 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558f297afd40 con 0x558f29868ca0 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.374+0000 7fd3d5600700 10 client.24632 handle_command_reply: tid=0 2024-06-02T15:38:59.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.374+0000 7fd3fc922b80 1 client.24632 shutdown 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 msgr2=0x558f29859000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 0x558f29859000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80a5940 tx=0x7fd3d80e8790 comp rx=0 tx=0).stop 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 shutdown_connections 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f29868ca0 0x558f2986a110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f29855ee0 0x558f29859540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2985bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/4173940521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f298555c0 0x558f29859000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3d5600700 0 client.24632 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 >> 172.21.15.90:0/4173940521 conn(0x558f2977db90 msgr2=0x558f29849d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 shutdown_connections 2024-06-02T15:38:59.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 1 -- 172.21.15.90:0/4173940521 wait complete. 2024-06-02T15:38:59.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.375+0000 7fd3fc922b80 20 client.24632 trim_cache size 0 max 16384 2024-06-02T15:38:59.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 Processor -- start 2024-06-02T15:38:59.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 -- start start 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-02T15:38:59.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-02T15:38:59.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-06-02T15:38:59.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-02T15:38:59.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:38:59 smithi090 ceph-mon[31595]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f29846990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f298569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558f29868450 con 0x558f298500f0 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f29868b20 con 0x558f29855ee0 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.398+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f298691f0 con 0x558f298555c0 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38080/0 (socket says 172.21.15.90:38080) 2024-06-02T15:38:59.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3eb400700 1 -- 172.21.15.90:0/3512082115 learned_addr learned my addr 172.21.15.90:0/3512082115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f298569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f29846990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 -- 172.21.15.90:0/3512082115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 msgr2=0x558f29846990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f29846990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 -- 172.21.15.90:0/3512082115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 msgr2=0x558f2984fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3ebe00700 1 -- 172.21.15.90:0/3512082115 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f29853b00 con 0x558f29855ee0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.399+0000 7fd3eb400700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.400+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f298569d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b070 tx=0x7fd3e401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f52332a1e1655b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.400+0000 7fd3d7e00700 1 -- 172.21.15.90:0/3512082115 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4023070 con 0x558f29855ee0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.400+0000 7fd3d7e00700 1 -- 172.21.15.90:0/3512082115 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4007990 con 0x558f29855ee0 2024-06-02T15:38:59.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.400+0000 7fd3d7e00700 1 -- 172.21.15.90:0/3512082115 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4005520 con 0x558f29855ee0 2024-06-02T15:38:59.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3512082115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 msgr2=0x558f298569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f298569d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b070 tx=0x7fd3e401f000 comp rx=0 tx=0).stop 2024-06-02T15:38:59.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3512082115 shutdown_connections 2024-06-02T15:38:59.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f298569d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298500f0 0x558f2984fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/3512082115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298555c0 0x558f29846990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3512082115 >> 172.21.15.90:0/3512082115 conn(0x558f2977db90 msgr2=0x558f298562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3512082115 shutdown_connections 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.401+0000 7fd3fc922b80 1 -- 172.21.15.90:0/3512082115 wait complete. 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.402+0000 7fd3fc922b80 1 -- start start 2024-06-02T15:38:59.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.402+0000 7fd3fc922b80 1 Processor -- start 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 0x558f29873590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2986e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558f29869f90 con 0x558f298555c0 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558f298698c0 con 0x558f29855ee0 2024-06-02T15:38:59.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f2986a660 con 0x558f298500f0 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3fc922b80 10 client.0 _ll_register_callbacks cb 0x558f29747c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 0x558f29873590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38094/0 (socket says 172.21.15.90:38094) 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.403+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1173292309 learned_addr learned my addr 172.21.15.90:0/1173292309 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3ebe00700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2986e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1173292309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 msgr2=0x558f29873590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 0x558f29873590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 msgr2=0x558f2986e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2986e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 -- 172.21.15.90:0/1173292309 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2986eb80 con 0x558f298555c0 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eb400700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 0x558f29873590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:38:59.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3eaa00700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80e0110 tx=0x7fd3d80e8800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c91db480564cb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3d5600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80f1070 con 0x558f298555c0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.404+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fd3d80e8320 con 0x558f298555c0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e4007990 con 0x558f298555c0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd3d80a9690 con 0x558f298555c0 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3fc922b80 10 client.14520 fetch_fsmap learned FSMap version 17 2024-06-02T15:38:59.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x558f2986eb80 con 0x558f298555c0 2024-06-02T15:38:59.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4007990 con 0x558f298555c0 2024-06-02T15:38:59.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fd3d80041d0 con 0x558f298555c0 2024-06-02T15:38:59.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.405+0000 7fd3fc922b80 10 client.14520 fetch_fsmap finished waiting for FSMap version 17 2024-06-02T15:38:59.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3fc922b80 10 client.14520 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:38:59.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3fc922b80 20 client.14520 populate_metadata read hostname 'smithi090' 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f298d94e0 0x558f298d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3fc922b80 4 client.14520 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "status"}) -- 0x558f297a9520 con 0x558f298d94e0 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3eb400700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f298d94e0 0x558f298d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3eb400700 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f298d94e0 0x558f298d81f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef90028170ecc5cd server_cookie=10a63c4160e3422a in_seq=0 out_seq=0 2024-06-02T15:38:59.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.406+0000 7fd3d5600700 10 client.14520 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:38:59.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3d5600700 1 -- 172.21.15.90:0/1173292309 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558f297a9520 con 0x558f298d94e0 2024-06-02T15:38:59.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3d5600700 10 client.14520 handle_command_reply: tid=0 2024-06-02T15:38:59.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 client.14520 shutdown 2024-06-02T15:38:59.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 msgr2=0x558f29873ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd3d80e0110 tx=0x7fd3d80e8800 comp rx=0 tx=0).stop 2024-06-02T15:38:59.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 shutdown_connections 2024-06-02T15:38:59.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558f298d94e0 0x558f298d81f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558f29855ee0 0x558f2986e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f298500f0 0x558f29873590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 --2- 172.21.15.90:0/1173292309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558f298555c0 0x558f29873ad0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 >> 172.21.15.90:0/1173292309 conn(0x558f2977db90 msgr2=0x558f29856b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3d5600700 0 client.14520 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:38:59.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 shutdown_connections 2024-06-02T15:38:59.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.407+0000 7fd3fc922b80 1 -- 172.21.15.90:0/1173292309 wait complete. 2024-06-02T15:38:59.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.408+0000 7fd3fc922b80 20 client.14520 trim_cache size 0 max 16384 2024-06-02T15:38:59.416 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24536, 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:38:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 39.927507317, 2024-06-02T15:38:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 17, 2024-06-02T15:38:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 88, 2024-06-02T15:38:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 85, 2024-06-02T15:38:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 62.378022545999997 2024-06-02T15:38:59.418 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:38:59.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.408+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 msgr2=0x7fd3d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.408+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 0x7fd3d00b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009fd0 tx=0x7fd3e4006210 comp rx=0 tx=0).stop 2024-06-02T15:38:59.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.408+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 msgr2=0x7fd3ec19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.408+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec19c3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e9f0 tx=0x7fd3d800ed00 comp rx=0 tx=0).stop 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 shutdown_connections 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3d00b2880 0x7fd3d00b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3ec10fff0 0x7fd3ec1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3ec06b250 0x7fd3ec19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 --2- 172.21.15.90:0/1043344110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ec06bb70 0x7fd3ec19c3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:38:59.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 >> 172.21.15.90:0/1043344110 conn(0x7fd3ec0793f0 msgr2=0x7fd3ec111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:38:59.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 shutdown_connections 2024-06-02T15:38:59.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:38:59.409+0000 7fd3f2000700 1 -- 172.21.15.90:0/1043344110 wait complete. 2024-06-02T15:38:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-02T15:38:59.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:38:59 smithi120 ceph-mon[36451]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T15:38:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: pgmap v346: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-06-02T15:38:59.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-06-02T15:38:59.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:38:59 smithi135 ceph-mon[36839]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T15:39:00.227 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-02T15:39:00.227 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:00.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-02T15:39:00.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:00 smithi120 ceph-mon[36451]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T15:39:00.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:00 smithi135 ceph-mon[36839]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T15:39:00.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:00 smithi090 ceph-mon[31595]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T15:39:00.654 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:01.345 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:01 smithi090 ceph-mon[31595]: pgmap v349: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:01 smithi120 ceph-mon[36451]: pgmap v349: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:01 smithi135 ceph-mon[36839]: pgmap v349: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.933+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1402276940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf44104fd0 msgr2=0x7fdf4410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.933+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1402276940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf44104fd0 0x7fdf4410d4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf300098f0 tx=0x7fdf30009c00 comp rx=0 tx=0).stop 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1402276940 shutdown_connections 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1402276940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf4410da90 0x7fdf4410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1402276940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 0x7fdf44104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1402276940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf44104fd0 0x7fdf4410d4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1402276940 >> 172.21.15.90:0/1402276940 conn(0x7fdf440fde60 msgr2=0x7fdf44100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:02.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1402276940 shutdown_connections 2024-06-02T15:39:02.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.934+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1402276940 wait complete. 2024-06-02T15:39:02.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 Processor -- start 2024-06-02T15:39:02.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 -- start start 2024-06-02T15:39:02.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 0x7fdf4419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:02.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 0x7fdf4419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 0x7fdf441a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44113630 con 0x7fdf44104fd0 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441134b0 con 0x7fdf4410da90 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.935+0000 7fdf4aa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf441137b0 con 0x7fdf441046b0 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf48c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 0x7fdf4419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 0x7fdf441a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf49600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 0x7fdf4419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf48c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 0x7fdf4419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50180/0 (socket says 172.21.15.90:50180) 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf48c00700 1 -- 172.21.15.90:0/1120413829 learned_addr learned my addr 172.21.15.90:0/1120413829 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 -- 172.21.15.90:0/1120413829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 msgr2=0x7fdf4419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 0x7fdf4419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 -- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 msgr2=0x7fdf4419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 0x7fdf4419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 -- 172.21.15.90:0/1120413829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf441a0ef0 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.936+0000 7fdf4a000700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 0x7fdf441a07b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00eb10 tx=0x7fdf3c00ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.937+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00cb70 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.937+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c0044e0 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.937+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf441a11e0 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.937+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c010430 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.937+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a8a60 con 0x7fdf4410da90 2024-06-02T15:39:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.938+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4406a230 con 0x7fdf4410da90 2024-06-02T15:39:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.940+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdf3c003680 con 0x7fdf4410da90 2024-06-02T15:39:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.942+0000 7fdf42000700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 0x7fdf2c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.942+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fdf3c0e2780 con 0x7fdf4410da90 2024-06-02T15:39:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.942+0000 7fdf49600700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 0x7fdf2c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:02.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.943+0000 7fdf49600700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 0x7fdf2c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009da0 tx=0x7fdf3800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:02.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:02.944+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf3c0aa170 con 0x7fdf4410da90 2024-06-02T15:39:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.230+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdf4419d240 con 0x7fdf4410da90 2024-06-02T15:39:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.231+0000 7fdf42000700 1 -- 172.21.15.90:0/1120413829 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fdf4419d240 con 0x7fdf4410da90 2024-06-02T15:39:03.229 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 msgr2=0x7fdf2c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 0x7fdf2c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009da0 tx=0x7fdf3800b040 comp rx=0 tx=0).stop 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 msgr2=0x7fdf441a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 0x7fdf441a07b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00eb10 tx=0x7fdf3c00ee20 comp rx=0 tx=0).stop 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 shutdown_connections 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdf2c0b2900 0x7fdf2c0b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdf4410da90 0x7fdf441a07b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf441046b0 0x7fdf4419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 --2- 172.21.15.90:0/1120413829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf44104fd0 0x7fdf4419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.234+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 >> 172.21.15.90:0/1120413829 conn(0x7fdf440fde60 msgr2=0x7fdf4410ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.235+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 shutdown_connections 2024-06-02T15:39:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:03.235+0000 7fdf4aa00700 1 -- 172.21.15.90:0/1120413829 wait complete. 2024-06-02T15:39:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:03 smithi120 ceph-mon[36451]: pgmap v350: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-02T15:39:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:03 smithi135 ceph-mon[36839]: pgmap v350: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-02T15:39:03.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:03 smithi090 ceph-mon[31595]: pgmap v350: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-02T15:39:04.120 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-02T15:39:04.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:04.121 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-06-02T15:39:04.175 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-02T15:39:04.175 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-02T15:39:04.175 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-06-02T15:39:04.175 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:39:04.176 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-02T15:39:04.177 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41922sec preferred_lft 41922sec 2024-06-02T15:39:04.177 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-06-02T15:39:04.177 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:39:04.178 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-02T15:39:04.178 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:39:04.178 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:39:04.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-06-02T15:39:04.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-06-02T15:39:04.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-06-02T15:39:04.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-02T15:39:04.179 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:39:04.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:39:04.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:04 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1120413829' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:39:04.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:04 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1120413829' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:39:04.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:04 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1120413829' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:39:04.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:04.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:04.685 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-02T15:39:04.752 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-06-02T15:39:04.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:04.756 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-06-02T15:39:04.812 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-06-02T15:39:04.812 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-02T15:39:04.812 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-02T15:39:04.812 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-02T15:39:04.813 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-02T15:39:04.813 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:39:04.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:39:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:05 smithi120 ceph-mon[36451]: pgmap v351: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:05 smithi135 ceph-mon[36839]: pgmap v351: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:05.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:05 smithi090 ceph-mon[31595]: pgmap v351: 129 pgs: 1 activating, 128 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:05.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:05.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:05.629 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-06-02T15:39:05.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:05.685 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-06-02T15:39:05.738 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:39:05.738 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:39:05.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:05.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-02T15:39:05.738 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:39:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:39:05.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:05.922 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-02T15:39:05.922 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:39:05.922 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:39:05.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-02T15:39:05.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-02T15:39:05.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-02T15:39:05.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-02T15:39:05.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-02T15:39:05.923 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:39:06.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:39:06.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:06.201 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:39:06.201 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:39:06.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-06-02T15:39:06.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-06-02T15:39:06.201 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:39:06.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:39:06.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:06.402 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-02T15:39:06.402 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T15:39:06.402 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:06.445 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-02T15:39:06.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T15:39:06.445 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:06.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-06-02T15:39:06.630 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T15:39:06.679 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-06-02T15:39:06.679 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-06-02T15:39:06.679 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-06-02T15:39:06.680 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-06-02T15:39:06.681 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/21ad5cba50d28407fc3bdf9174bfe214182711021391a67bb045d916c9f4ac8e/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d3d971600d5b370d32b2e863da5d1fdaae207c7ae8a547f2773e1a5ef3a5c342/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/68a49acf4acff8e4e1b476dd1865ea8f205ec2ea90cf12afe6d9ec57220e9a9b/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/10d8d7a1f06da77f7243653e709e93d3326ea607a0b481f07e710566a3dc44b1/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c76e748ff8e3e422d289d3041e52f7e0ccb57f3cecdca6240f228d9ac512d893/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/0ca4369a962680da4de7919769eb60f24fc0c467ecfffab6ef946298ba78a32d/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ad79fd76666fde0c507aebff527d9043a0790ee1dee6760a4ff1656cb966dcb9/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/718bf2b53975015dac382e392086ecba775e72255210a4cd8bb468a78fc7c8d7/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/55335265f1674b88591397334355c29212e2aef3cd74c8f9383053c68bc893a5/merged 2024-06-02T15:39:06.682 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/6c8e2834f59f7f2867d0765c4cf630682cac653426d0e18d3dec58a5426d8619/merged 2024-06-02T15:39:06.683 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-06-02T15:39:06.683 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:06.683 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:06.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:06.684 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-06-02T15:39:06.729 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-02T15:39:06.729 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-02T15:39:06.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-06-02T15:39:06.851 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T15:39:06.897 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-06-02T15:39:06.897 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-06-02T15:39:06.897 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-06-02T15:39:06.897 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-06-02T15:39:06.898 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-06-02T15:39:06.899 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/21ad5cba50d28407fc3bdf9174bfe214182711021391a67bb045d916c9f4ac8e/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d3d971600d5b370d32b2e863da5d1fdaae207c7ae8a547f2773e1a5ef3a5c342/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/68a49acf4acff8e4e1b476dd1865ea8f205ec2ea90cf12afe6d9ec57220e9a9b/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/10d8d7a1f06da77f7243653e709e93d3326ea607a0b481f07e710566a3dc44b1/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c76e748ff8e3e422d289d3041e52f7e0ccb57f3cecdca6240f228d9ac512d893/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/0ca4369a962680da4de7919769eb60f24fc0c467ecfffab6ef946298ba78a32d/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ad79fd76666fde0c507aebff527d9043a0790ee1dee6760a4ff1656cb966dcb9/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/718bf2b53975015dac382e392086ecba775e72255210a4cd8bb468a78fc7c8d7/merged 2024-06-02T15:39:06.900 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/55335265f1674b88591397334355c29212e2aef3cd74c8f9383053c68bc893a5/merged 2024-06-02T15:39:06.901 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/6c8e2834f59f7f2867d0765c4cf630682cac653426d0e18d3dec58a5426d8619/merged 2024-06-02T15:39:06.901 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-06-02T15:39:06.901 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:06.901 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:06.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:06.902 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-06-02T15:39:06.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:2024-06-02T15:39:06.985+0000 7ff9a200c480 -1 init, newargv = 0x55c37f03f720 newargc=15 2024-06-02T15:39:06.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[77392]: starting ceph client 2024-06-02T15:39:06.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[77392]: starting fuse 2024-06-02T15:39:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:07 smithi120 ceph-mon[36451]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:39:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:07 smithi135 ceph-mon[36839]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:39:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:07 smithi090 ceph-mon[31595]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 274 B/s, 0 objects/s recovering 2024-06-02T15:39:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:39:08.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-06-02T15:39:08.132 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T15:39:08.181 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-06-02T15:39:08.182 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-06-02T15:39:08.183 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/21ad5cba50d28407fc3bdf9174bfe214182711021391a67bb045d916c9f4ac8e/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d3d971600d5b370d32b2e863da5d1fdaae207c7ae8a547f2773e1a5ef3a5c342/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/68a49acf4acff8e4e1b476dd1865ea8f205ec2ea90cf12afe6d9ec57220e9a9b/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/10d8d7a1f06da77f7243653e709e93d3326ea607a0b481f07e710566a3dc44b1/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c76e748ff8e3e422d289d3041e52f7e0ccb57f3cecdca6240f228d9ac512d893/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/0ca4369a962680da4de7919769eb60f24fc0c467ecfffab6ef946298ba78a32d/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ad79fd76666fde0c507aebff527d9043a0790ee1dee6760a4ff1656cb966dcb9/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/718bf2b53975015dac382e392086ecba775e72255210a4cd8bb468a78fc7c8d7/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/55335265f1674b88591397334355c29212e2aef3cd74c8f9383053c68bc893a5/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/6c8e2834f59f7f2867d0765c4cf630682cac653426d0e18d3dec58a5426d8619/merged 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-06-02T15:39:08.184 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:08.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:39:08.185 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:08.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:08.185 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-06-02T15:39:08.226 INFO:teuthology.orchestra.run.smithi090.stdout:119 2024-06-02T15:39:08.226 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-02T15:39:08.226 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-06-02T15:39:08.227 DEBUG:teuthology.orchestra.run.smithi090:> 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.admin")) 2024-06-02T15:39:08.228 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-06-02T15:39:08.418 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.admin.77392.asok 2024-06-02T15:39:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:08.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.77392.asok 2024-06-02T15:39:08.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:08.435 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.77392.asok status 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "b7328fed0e2e9ab66b09f8134f3cebfc71214504", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3882-gb7328fed (b7328fed0e2e9ab66b09f8134f3cebfc71214504) squid (dev)", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "admin", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "77392", 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:39:08.661 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 14529, 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 14529 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3264100455 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:39:08.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3264100455 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.14529 192.168.0.1:0/3264100455", 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/3264100455", 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 17, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 88, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-06-02T15:39:08.663 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:39:08.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:08.676 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-06-02T15:39:08.676 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:08.722 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-06-02T15:39:08.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:08.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:08.723 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:08.832 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-02T15:39:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-02T15:39:08.953 INFO:teuthology.orchestra.run.smithi090.stdout:# file: . 2024-06-02T15:39:08.953 INFO:teuthology.orchestra.run.smithi090.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-02T15:39:08.953 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:39:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:39:08.964 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:39:08.964 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T15:39:09.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-02T15:39:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:09 smithi090 ceph-mon[31595]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 214 B/s wr, 0 op/s; 230 B/s, 0 objects/s recovering 2024-06-02T15:39:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:09 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.458 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:09 smithi120 ceph-mon[36451]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 214 B/s wr, 0 op/s; 230 B/s, 0 objects/s recovering 2024-06-02T15:39:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:09 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:09 smithi135 ceph-mon[36839]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 214 B/s wr, 0 op/s; 230 B/s, 0 objects/s recovering 2024-06-02T15:39:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:09.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:09 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:10.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:10 smithi090 ceph-mon[31595]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 0 op/s; 208 B/s, 0 objects/s recovering 2024-06-02T15:39:10.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:10.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:10 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:11.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:10 smithi135 ceph-mon[36839]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 0 op/s; 208 B/s, 0 objects/s recovering 2024-06-02T15:39:11.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:11.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:10 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:11.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:10 smithi120 ceph-mon[36451]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 194 B/s wr, 0 op/s; 208 B/s, 0 objects/s recovering 2024-06-02T15:39:11.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:11.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:10 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:11.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 -- 172.21.15.90:0/88207533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f01095a0 msgr2=0x7f58f0109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:11.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 --2- 172.21.15.90:0/88207533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f01095a0 0x7f58f0109980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58e000b0a0 tx=0x7f58e000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 -- 172.21.15.90:0/88207533 shutdown_connections 2024-06-02T15:39:11.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 --2- 172.21.15.90:0/88207533 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006c470 0x7f58f01103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 --2- 172.21.15.90:0/88207533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f01095a0 0x7f58f0109980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 --2- 172.21.15.90:0/88207533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f006bab0 0x7f58f006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.722+0000 7f58f5e00700 1 -- 172.21.15.90:0/88207533 >> 172.21.15.90:0/88207533 conn(0x7f58f00793b0 msgr2=0x7f58f00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.723+0000 7f58f5e00700 1 -- 172.21.15.90:0/88207533 shutdown_connections 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.723+0000 7f58f5e00700 1 -- 172.21.15.90:0/88207533 wait complete. 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 Processor -- start 2024-06-02T15:39:11.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 -- start start 2024-06-02T15:39:11.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 0x7f58f019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f01095a0 0x7f58f0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113700 con 0x7f58f01095a0 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113580 con 0x7f58f006bab0 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.724+0000 7f58f5e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0113880 con 0x7f58f006c470 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41512/0 (socket says 172.21.15.90:41512) 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 -- 172.21.15.90:0/3370743672 learned_addr learned my addr 172.21.15.90:0/3370743672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58ef400700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f01095a0 0x7f58f0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58eea00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 0x7f58f019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:11.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 -- 172.21.15.90:0/3370743672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 msgr2=0x7f58f019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 0x7f58f019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 -- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f01095a0 msgr2=0x7f58f0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f01095a0 0x7f58f0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 -- 172.21.15.90:0/3370743672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f019a1a0 con 0x7f58f006c470 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58eea00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 0x7f58f019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.725+0000 7f58e6000700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58d800d980 tx=0x7f58d800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.726+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8016070 con 0x7f58f006c470 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.726+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f019a490 con 0x7f58f006c470 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.726+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d8004830 con 0x7f58f006c470 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.726+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8005540 con 0x7f58f006c470 2024-06-02T15:39:11.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.726+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a6750 con 0x7f58f006c470 2024-06-02T15:39:11.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.727+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f010ba80 con 0x7f58f006c470 2024-06-02T15:39:11.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.729+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f58d8004220 con 0x7f58f006c470 2024-06-02T15:39:11.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.731+0000 7f58f4a00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 0x7f58d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:11.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.731+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f58d80e2390 con 0x7f58f006c470 2024-06-02T15:39:11.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.731+0000 7f58eea00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 0x7f58d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:11.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.732+0000 7f58eea00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 0x7f58d40b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58e00062a0 tx=0x7f58e0006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:11.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.735+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58d801f020 con 0x7f58f006c470 2024-06-02T15:39:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.990+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58f006a230 con 0x7f58f006c470 2024-06-02T15:39:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.990+0000 7f58f4a00700 1 -- 172.21.15.90:0/3370743672 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f58d80a9cf0 con 0x7f58f006c470 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2134019795 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3164241316 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/232748990 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/828520669 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1506181662 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:11.989 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:11.990 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:11.990 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:11.990 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1946492510 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:11.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 msgr2=0x7f58d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:11.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 0x7f58d40b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58e00062a0 tx=0x7f58e0006210 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 msgr2=0x7f58f019fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58d800d980 tx=0x7f58d800dc90 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 shutdown_connections 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58d40b2880 0x7f58d40b4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f01095a0 0x7f58f0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58f006bab0 0x7f58f019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 --2- 172.21.15.90:0/3370743672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f006c470 0x7f58f019fdc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.993+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 >> 172.21.15.90:0/3370743672 conn(0x7f58f00793b0 msgr2=0x7f58f0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.994+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 shutdown_connections 2024-06-02T15:39:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:11.994+0000 7f58f5e00700 1 -- 172.21.15.90:0/3370743672 wait complete. 2024-06-02T15:39:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-06-02T15:39:12.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:39:12.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:39:12.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:11 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:12.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:39:12.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:39:12.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:11 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:12.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:39:12.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:39:12.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:11 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:39:12.874 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:39:12.874 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T15:39:12.930 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-02T15:39:13.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:12 smithi135 ceph-mon[36839]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-06-02T15:39:13.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:12 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3370743672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:13.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:12 smithi090 ceph-mon[31595]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-06-02T15:39:13.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:12 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3370743672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:13.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:12 smithi120 ceph-mon[36451]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s; 182 B/s, 0 objects/s recovering 2024-06-02T15:39:13.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:12 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3370743672' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:13.366 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:14 smithi135 ceph-mon[36839]: pgmap v356: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-02T15:39:15.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:14 smithi090 ceph-mon[31595]: pgmap v356: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-02T15:39:15.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:14 smithi120 ceph-mon[36451]: pgmap v356: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-06-02T15:39:15.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.596+0000 7f8835200700 1 -- 172.21.15.90:0/3542327326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 msgr2=0x7f8830103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.596+0000 7f8835200700 1 --2- 172.21.15.90:0/3542327326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830103c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88200098f0 tx=0x7f8820009c00 comp rx=0 tx=0).stop 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 -- 172.21.15.90:0/3542327326 shutdown_connections 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 --2- 172.21.15.90:0/3542327326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 0x7f88301104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 --2- 172.21.15.90:0/3542327326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830103c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 --2- 172.21.15.90:0/3542327326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 0x7f88301081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 -- 172.21.15.90:0/3542327326 >> 172.21.15.90:0/3542327326 conn(0x7f883007a5d0 msgr2=0x7f883007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 -- 172.21.15.90:0/3542327326 shutdown_connections 2024-06-02T15:39:15.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.597+0000 7f8835200700 1 -- 172.21.15.90:0/3542327326 wait complete. 2024-06-02T15:39:15.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.599+0000 7f8835200700 1 Processor -- start 2024-06-02T15:39:15.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.599+0000 7f8835200700 1 -- start start 2024-06-02T15:39:15.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:15.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 0x7f8830075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:15.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 0x7f88300762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:15.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8830113770 con 0x7f8830108920 2024-06-02T15:39:15.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88301135f0 con 0x7f8830104200 2024-06-02T15:39:15.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f8835200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88301138f0 con 0x7f8830103850 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f882de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f882de00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41526/0 (socket says 172.21.15.90:41526) 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f882de00700 1 -- 172.21.15.90:0/2466214622 learned_addr learned my addr 172.21.15.90:0/2466214622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.600+0000 7f882d400700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 0x7f8830075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882e800700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 0x7f88300762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 -- 172.21.15.90:0/2466214622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 msgr2=0x7f8830075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 0x7f8830075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 -- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 msgr2=0x7f88300762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 0x7f88300762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 -- 172.21.15.90:0/2466214622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88301a81b0 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f882de00700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8820009230 tx=0x7f882000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.601+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820005140 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.602+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88301a8440 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.602+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882000b560 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.602+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820003970 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.602+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301a88f0 con 0x7f8830103850 2024-06-02T15:39:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.603+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883010bb60 con 0x7f8830103850 2024-06-02T15:39:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.605+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f882000ba50 con 0x7f8830103850 2024-06-02T15:39:15.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.607+0000 7f8826a00700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 0x7f88140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:15.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.607+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f88200e3630 con 0x7f8830103850 2024-06-02T15:39:15.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.607+0000 7f882d400700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 0x7f88140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:15.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.608+0000 7f882d400700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 0x7f88140b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8830103200 tx=0x7f8818009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:15.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.609+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f882005fdf0 con 0x7f8830103850 2024-06-02T15:39:15.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.856+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f883010b540 con 0x7f8830103850 2024-06-02T15:39:15.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.857+0000 7f8826a00700 1 -- 172.21.15.90:0/2466214622 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f8820013030 con 0x7f8830103850 2024-06-02T15:39:15.856 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2134019795 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:15.856 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3164241316 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:15.856 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:15.856 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/232748990 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/828520669 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1506181662 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:39:15.857 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1946492510 2024-06-03T15:29:57.531987+0000 2024-06-02T15:39:15.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 msgr2=0x7f88140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 0x7f88140b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8830103200 tx=0x7f8818009310 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 msgr2=0x7f8830075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8820009230 tx=0x7f882000fa70 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 shutdown_connections 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88140b2900 0x7f88140b4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8830108920 0x7f88300762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8830103850 0x7f8830075840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 --2- 172.21.15.90:0/2466214622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8830104200 0x7f8830075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 >> 172.21.15.90:0/2466214622 conn(0x7f883007a5d0 msgr2=0x7f8830101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 shutdown_connections 2024-06-02T15:39:15.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:15.861+0000 7f8835200700 1 -- 172.21.15.90:0/2466214622 wait complete. 2024-06-02T15:39:15.861 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-06-02T15:39:16.657 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-06-02T15:39:16.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:16.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-02T15:39:16.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[77392]: fuse finished with error 0 and tester_r 0 2024-06-02T15:39:16.745 INFO:teuthology.orchestra.run:waiting for 300 2024-06-02T15:39:16.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:39:16.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:39:16.783 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-02T15:39:16.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:16 smithi090 ceph-mon[31595]: pgmap v357: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-02T15:39:16.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2466214622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:16.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:39:17.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:16 smithi120 ceph-mon[36451]: pgmap v357: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-02T15:39:17.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:16 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2466214622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:17.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:16 smithi135 ceph-mon[36839]: pgmap v357: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-02T15:39:17.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:16 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2466214622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:39:17.390 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:19.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:18 smithi090 ceph-mon[31595]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-02T15:39:19.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:18 smithi120 ceph-mon[36451]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-02T15:39:19.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:18 smithi135 ceph-mon[36839]: pgmap v358: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-06-02T15:39:19.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 -- 172.21.15.90:0/413579700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81041a0 msgr2=0x7f2fa81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:19.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 --2- 172.21.15.90:0/413579700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81041a0 0x7f2fa81081c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009990 tx=0x7f2f90009ca0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 -- 172.21.15.90:0/413579700 shutdown_connections 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 --2- 172.21.15.90:0/413579700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81088f0 0x7f2fa8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 --2- 172.21.15.90:0/413579700 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 --2- 172.21.15.90:0/413579700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81041a0 0x7f2fa81081c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 -- 172.21.15.90:0/413579700 >> 172.21.15.90:0/413579700 conn(0x7f2fa807a6a0 msgr2=0x7f2fa807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 -- 172.21.15.90:0/413579700 shutdown_connections 2024-06-02T15:39:19.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.611+0000 7f2fade00700 1 -- 172.21.15.90:0/413579700 wait complete. 2024-06-02T15:39:19.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 Processor -- start 2024-06-02T15:39:19.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 -- start start 2024-06-02T15:39:19.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81088f0 0x7f2fa80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8113720 con 0x7f2fa81041a0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81135a0 con 0x7f2fa81037f0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2fade00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81138a0 con 0x7f2fa81088f0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.613+0000 7f2f9e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57012/0 (socket says 172.21.15.90:57012) 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 -- 172.21.15.90:0/4065939611 learned_addr learned my addr 172.21.15.90:0/4065939611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2fa6a00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2fa7400700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81088f0 0x7f2fa80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 -- 172.21.15.90:0/4065939611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81088f0 msgr2=0x7f2fa80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81088f0 0x7f2fa80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 -- 172.21.15.90:0/4065939611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 msgr2=0x7f2fa8075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 -- 172.21.15.90:0/4065939611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa81a8120 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2fa6a00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.614+0000 7f2f9e000700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000c00 tx=0x7f2f900057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.615+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90027070 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.615+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9000b610 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.615+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900173f0 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.615+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa81a83b0 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.615+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa81a87b0 con 0x7f2fa81041a0 2024-06-02T15:39:19.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.616+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa810bac0 con 0x7f2fa81041a0 2024-06-02T15:39:19.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.618+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2f9002a070 con 0x7f2fa81041a0 2024-06-02T15:39:19.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.620+0000 7f2faca00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 0x7f2f8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:19.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.620+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2f900e33e0 con 0x7f2fa81041a0 2024-06-02T15:39:19.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.620+0000 7f2fa6a00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 0x7f2f8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:19.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.621+0000 7f2fa6a00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 0x7f2f8c0b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f98003e30 tx=0x7f2f9800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.625+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f90013070 con 0x7f2fa81041a0 2024-06-02T15:39:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.861+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fa802e6c0 con 0x7f2fa81041a0 2024-06-02T15:39:19.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.866+0000 7f2faca00700 1 -- 172.21.15.90:0/4065939611 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24611 (secure 0 0 0) 0x7f2f900aae30 con 0x7f2fa81041a0 2024-06-02T15:39:19.867 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:39:19.867 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:38:59.160855+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:39:19.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 msgr2=0x7f2f8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 0x7f2f8c0b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f98003e30 tx=0x7f2f9800b040 comp rx=0 tx=0).stop 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 msgr2=0x7f2fa8075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f90000c00 tx=0x7f2f900057c0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 shutdown_connections 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2f8c0b2880 0x7f2f8c0b4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fa81088f0 0x7f2fa80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fa81037f0 0x7f2fa8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 --2- 172.21.15.90:0/4065939611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa81041a0 0x7f2fa8075d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:19.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 >> 172.21.15.90:0/4065939611 conn(0x7f2fa807a6a0 msgr2=0x7f2fa8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:19.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 shutdown_connections 2024-06-02T15:39:19.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:19.870+0000 7f2fade00700 1 -- 172.21.15.90:0/4065939611 wait complete. 2024-06-02T15:39:20.809 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:39:21.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:20 smithi090 ceph-mon[31595]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:21.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4065939611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:21.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:20 smithi120 ceph-mon[36451]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:21.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4065939611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:21.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:20 smithi135 ceph-mon[36839]: pgmap v359: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:21.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4065939611' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:23.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:22 smithi090 ceph-mon[31595]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:23.110 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:22 smithi120 ceph-mon[36451]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:23.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:22 smithi135 ceph-mon[36839]: pgmap v360: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:23.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 -- 172.21.15.90:0/2617990792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60106120 msgr2=0x7f9d60112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:23.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 --2- 172.21.15.90:0/2617990792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60106120 0x7f9d60112630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b0a0 tx=0x7f9d5800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 -- 172.21.15.90:0/2617990792 shutdown_connections 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 --2- 172.21.15.90:0/2617990792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60106120 0x7f9d60112630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 --2- 172.21.15.90:0/2617990792 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 0x7f9d6010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 --2- 172.21.15.90:0/2617990792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60105760 0x7f9d60105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 -- 172.21.15.90:0/2617990792 >> 172.21.15.90:0/2617990792 conn(0x7f9d60101250 msgr2=0x7f9d60103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 -- 172.21.15.90:0/2617990792 shutdown_connections 2024-06-02T15:39:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.569+0000 7f9d68600700 1 -- 172.21.15.90:0/2617990792 wait complete. 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 Processor -- start 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 -- start start 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60106120 0x7f9d6019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 0x7f9d601a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601159f0 con 0x7f9d60106120 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60115870 con 0x7f9d60105760 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.571+0000 7f9d68600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60115b70 con 0x7f9d6010b760 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60106120 0x7f9d6019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60648/0 (socket says 172.21.15.90:60648) 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 -- 172.21.15.90:0/1296120121 learned_addr learned my addr 172.21.15.90:0/1296120121 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:23.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d66600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 0x7f9d601a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 -- 172.21.15.90:0/1296120121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 msgr2=0x7f9d601a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 0x7f9d601a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 -- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60106120 msgr2=0x7f9d6019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60106120 0x7f9d6019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.572+0000 7f9d65c00700 1 -- 172.21.15.90:0/1296120121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601a0e20 con 0x7f9d60105760 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d65c00700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9c0 tx=0x7f9d5400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400cbd0 con 0x7f9d60105760 2024-06-02T15:39:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5400ee60 con 0x7f9d60105760 2024-06-02T15:39:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601a1110 con 0x7f9d60105760 2024-06-02T15:39:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540055c0 con 0x7f9d60105760 2024-06-02T15:39:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.573+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601a8990 con 0x7f9d60105760 2024-06-02T15:39:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.574+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6010dcf0 con 0x7f9d60105760 2024-06-02T15:39:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.576+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9d54004750 con 0x7f9d60105760 2024-06-02T15:39:23.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.577+0000 7f9d5e000700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 0x7f9d480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:23.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.578+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9d540e2750 con 0x7f9d60105760 2024-06-02T15:39:23.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.578+0000 7f9d65200700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 0x7f9d480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:23.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.579+0000 7f9d65200700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 0x7f9d480b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d601038c0 tx=0x7f9d4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:23.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.582+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d540aa0b0 con 0x7f9d60105760 2024-06-02T15:39:23.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.812+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d6019d170 con 0x7f9d60105760 2024-06-02T15:39:23.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.814+0000 7f9d5e000700 1 -- 172.21.15.90:0/1296120121 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24611 (secure 0 0 0) 0x7f9d540a99b0 con 0x7f9d60105760 2024-06-02T15:39:23.813 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:39:23.813 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:38:59.160855+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:39:23.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 msgr2=0x7f9d480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:23.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 0x7f9d480b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d601038c0 tx=0x7f9d4c009260 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 msgr2=0x7f9d6019be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9c0 tx=0x7f9d5400ecd0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 shutdown_connections 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9d480b2900 0x7f9d480b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d6010b760 0x7f9d601a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d60105760 0x7f9d6019be60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 --2- 172.21.15.90:0/1296120121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d60106120 0x7f9d6019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 >> 172.21.15.90:0/1296120121 conn(0x7f9d60101250 msgr2=0x7f9d60101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 shutdown_connections 2024-06-02T15:39:23.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:23.818+0000 7f9d68600700 1 -- 172.21.15.90:0/1296120121 wait complete. 2024-06-02T15:39:24.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:23 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1296120121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:24.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:23 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1296120121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:24.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:23 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1296120121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:39:24.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-02T15:39:25.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:24 smithi090 ceph-mon[31595]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:25.216 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:25.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:24 smithi120 ceph-mon[36451]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:25.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:24 smithi135 ceph-mon[36839]: pgmap v361: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-06-02T15:39:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:26 smithi090 ceph-mon[31595]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:26 smithi120 ceph-mon[36451]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:27.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:26 smithi135 ceph-mon[36839]: pgmap v362: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:27.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 -- 172.21.15.90:0/3568954406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 msgr2=0x7ff8f410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 --2- 172.21.15.90:0/3568954406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f410e7d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00099c0 tx=0x7ff8e0009cd0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 -- 172.21.15.90:0/3568954406 shutdown_connections 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 --2- 172.21.15.90:0/3568954406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f410eeb0 0x7ff8f4116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 --2- 172.21.15.90:0/3568954406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f4109dd0 0x7ff8f410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 --2- 172.21.15.90:0/3568954406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f410e7d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.445+0000 7ff8fb000700 1 -- 172.21.15.90:0/3568954406 >> 172.21.15.90:0/3568954406 conn(0x7ff8f4076d10 msgr2=0x7ff8f4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.446+0000 7ff8fb000700 1 -- 172.21.15.90:0/3568954406 shutdown_connections 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.446+0000 7ff8fb000700 1 -- 172.21.15.90:0/3568954406 wait complete. 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 Processor -- start 2024-06-02T15:39:27.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 -- start start 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 0x7ff8f41a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4119d40 con 0x7ff8f410a780 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4119bc0 con 0x7ff8f410eeb0 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.447+0000 7ff8fb000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4119ec0 con 0x7ff8f4109dd0 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:27.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:59342/0 (socket says 172.21.15.90:59342) 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55126/0 (socket says 172.21.15.90:55126) 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3e00700 1 -- 172.21.15.90:0/155631329 learned_addr learned my addr 172.21.15.90:0/155631329 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f9000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 0x7ff8f41a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.448+0000 7ff8f3e00700 1 -- 172.21.15.90:0/155631329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 msgr2=0x7ff8f41a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3e00700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 0x7ff8f41a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3e00700 1 -- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 msgr2=0x7ff8f41a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3e00700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3e00700 1 -- 172.21.15.90:0/155631329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41a51d0 con 0x7ff8f4109dd0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3400700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f9000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 0x7ff8f41a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f3e00700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9a0 tx=0x7ff8e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400ca10 con 0x7ff8f4109dd0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41a54c0 con 0x7ff8f4109dd0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e400cbb0 con 0x7ff8f4109dd0 2024-06-02T15:39:27.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4018680 con 0x7ff8f4109dd0 2024-06-02T15:39:27.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.449+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41acd40 con 0x7ff8f4109dd0 2024-06-02T15:39:27.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.451+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f406a230 con 0x7ff8f4109dd0 2024-06-02T15:39:27.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.453+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff8e4010430 con 0x7ff8f4109dd0 2024-06-02T15:39:27.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.454+0000 7ff8f0c00700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 0x7ff8d80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:27.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.455+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff8e40e2a00 con 0x7ff8f4109dd0 2024-06-02T15:39:27.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.455+0000 7ff8f3400700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 0x7ff8d80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:27.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.455+0000 7ff8f3400700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 0x7ff8d80b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:27.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.456+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8e40aa360 con 0x7ff8f4109dd0 2024-06-02T15:39:27.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.740+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff8f402e560 con 0x7ff8f4109dd0 2024-06-02T15:39:27.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.868+0000 7ff8f0c00700 1 -- 172.21.15.90:0/155631329 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7ff8e40b23e0 con 0x7ff8f4109dd0 2024-06-02T15:39:27.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 msgr2=0x7ff8d80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:27.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 0x7ff8d80b4dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0000c00 tx=0x7ff8e0005fb0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 msgr2=0x7ff8f41a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400e9a0 tx=0x7ff8e400ecb0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 shutdown_connections 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff8d80b2900 0x7ff8d80b4dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff8f410eeb0 0x7ff8f41a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8f4109dd0 0x7ff8f41a0210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.872+0000 7ff8fb000700 1 --2- 172.21.15.90:0/155631329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8f410a780 0x7ff8f41a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.873+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 >> 172.21.15.90:0/155631329 conn(0x7ff8f4076d10 msgr2=0x7ff8f40772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.873+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 shutdown_connections 2024-06-02T15:39:27.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:27.873+0000 7ff8fb000700 1 -- 172.21.15.90:0/155631329 wait complete. 2024-06-02T15:39:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:27 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/155631329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:27 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:27 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/155631329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:27 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:27 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/155631329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:27 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T15:39:28.741 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-06-02T15:39:28.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-06-02T15:39:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:28 smithi090 ceph-mon[31595]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:28 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T15:39:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:28 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:29.195 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:28 smithi120 ceph-mon[36451]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:28 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T15:39:29.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:28 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:28 smithi135 ceph-mon[36839]: pgmap v363: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-02T15:39:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:28 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T15:39:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:28 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:30 smithi090 ceph-mon[31595]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:31.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:30 smithi120 ceph-mon[36451]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:31.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:30 smithi135 ceph-mon[36839]: pgmap v364: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 -- 172.21.15.90:0/1736980490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c103e20 msgr2=0x7f201c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 --2- 172.21.15.90:0/1736980490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c103e20 0x7f201c110330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f201400b0a0 tx=0x7f201400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 -- 172.21.15.90:0/1736980490 shutdown_connections 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 --2- 172.21.15.90:0/1736980490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c103e20 0x7f201c110330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 --2- 172.21.15.90:0/1736980490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c1094f0 0x7f201c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 --2- 172.21.15.90:0/1736980490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 -- 172.21.15.90:0/1736980490 >> 172.21.15.90:0/1736980490 conn(0x7f201c0fef50 msgr2=0x7f201c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 -- 172.21.15.90:0/1736980490 shutdown_connections 2024-06-02T15:39:31.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.469+0000 7f2021400700 1 -- 172.21.15.90:0/1736980490 wait complete. 2024-06-02T15:39:31.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 Processor -- start 2024-06-02T15:39:31.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 -- start start 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c103e20 0x7f201c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c1094f0 0x7f201c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f201c113680 con 0x7f201c1094f0 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f201c113500 con 0x7f201c103e20 2024-06-02T15:39:31.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.471+0000 7f2021400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f201c113800 con 0x7f201c103460 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f2020a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c1094f0 0x7f201c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58152/0 (socket says 172.21.15.90:58152) 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 -- 172.21.15.90:0/605002087 learned_addr learned my addr 172.21.15.90:0/605002087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201b400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c103e20 0x7f201c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 -- 172.21.15.90:0/605002087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c103e20 msgr2=0x7f201c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c103e20 0x7f201c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 -- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c1094f0 msgr2=0x7f201c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c1094f0 0x7f201c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 -- 172.21.15.90:0/605002087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c19a100 con 0x7f201c103460 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.472+0000 7f201be00700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d980 tx=0x7f200c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.473+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c016070 con 0x7f201c103460 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.473+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c19a3f0 con 0x7f201c103460 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.473+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c004830 con 0x7f201c103460 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.473+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c005540 con 0x7f201c103460 2024-06-02T15:39:31.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.473+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c1a6660 con 0x7f201c103460 2024-06-02T15:39:31.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.474+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c06a230 con 0x7f201c103460 2024-06-02T15:39:31.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.476+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f200c004220 con 0x7f201c103460 2024-06-02T15:39:31.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.477+0000 7f2018c00700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 0x7f20000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:31.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.478+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f200c0e2900 con 0x7f201c103460 2024-06-02T15:39:31.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.478+0000 7f201b400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 0x7f20000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:31.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.478+0000 7f201b400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 0x7f20000b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f201c108b20 tx=0x7f2008009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:31.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.480+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f200c01f020 con 0x7f201c103460 2024-06-02T15:39:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.741+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f201c19ae80 con 0x7f201c103460 2024-06-02T15:39:31.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.886+0000 7f2018c00700 1 -- 172.21.15.90:0/605002087 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f200c0aa260 con 0x7f201c103460 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 msgr2=0x7f20000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 0x7f20000b4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f201c108b20 tx=0x7f2008009260 comp rx=0 tx=0).stop 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 msgr2=0x7f201c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f200c00d980 tx=0x7f200c00dc90 comp rx=0 tx=0).stop 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 shutdown_connections 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f20000b2880 0x7f20000b4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f201c1094f0 0x7f201c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f201c103460 0x7f201c19f790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 --2- 172.21.15.90:0/605002087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f201c103e20 0x7f201c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:31.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 >> 172.21.15.90:0/605002087 conn(0x7f201c0fef50 msgr2=0x7f201c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:31.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 shutdown_connections 2024-06-02T15:39:31.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:31.890+0000 7f2021400700 1 -- 172.21.15.90:0/605002087 wait complete. 2024-06-02T15:39:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:31 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/605002087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:31 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:31 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/605002087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:31 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:31 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/605002087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:31 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-06-02T15:39:32.852 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-02T15:39:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T15:39:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T15:39:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-02T15:39:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T15:39:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-02T15:39:33.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:32 smithi120 ceph-mon[36451]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-02T15:39:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:39:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T15:39:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-02T15:39:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T15:39:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-02T15:39:33.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:32 smithi135 ceph-mon[36839]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-02T15:39:33.322 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-06-02T15:39:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T15:39:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T15:39:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-06-02T15:39:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:32 smithi090 ceph-mon[31595]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-06-02T15:39:33.807 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:39:33.311+0000 7f024696e700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:active 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-02T15:39:34.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:33 smithi090 ceph-mon[31595]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-02T15:39:34.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:active 2024-06-02T15:39:34.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T15:39:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T15:39:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:39:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-02T15:39:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-02T15:39:34.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:33 smithi120 ceph-mon[36451]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] up:active 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-06-02T15:39:34.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:33 smithi135 ceph-mon[36839]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-06-02T15:39:35.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:34 smithi090 ceph-mon[31595]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-02T15:39:35.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:34 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:active 2024-06-02T15:39:35.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:34 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-02T15:39:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:34 smithi120 ceph-mon[36451]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-02T15:39:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:34 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:active 2024-06-02T15:39:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:34 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-02T15:39:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:34 smithi135 ceph-mon[36839]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-02T15:39:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:34 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] up:active 2024-06-02T15:39:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:34 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-06-02T15:39:35.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 -- 172.21.15.90:0/3900876074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881810ff70 msgr2=0x7f8818112360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:35.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 --2- 172.21.15.90:0/3900876074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881810ff70 0x7f8818112360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f881400b0a0 tx=0x7f881400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 -- 172.21.15.90:0/3900876074 shutdown_connections 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 --2- 172.21.15.90:0/3900876074 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f881810ff70 0x7f8818112360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 --2- 172.21.15.90:0/3900876074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f8818075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 --2- 172.21.15.90:0/3900876074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8818076090 0x7f881810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 -- 172.21.15.90:0/3900876074 >> 172.21.15.90:0/3900876074 conn(0x7f881807a6f0 msgr2=0x7f881807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.791+0000 7f8820800700 1 -- 172.21.15.90:0/3900876074 shutdown_connections 2024-06-02T15:39:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.792+0000 7f8820800700 1 -- 172.21.15.90:0/3900876074 wait complete. 2024-06-02T15:39:35.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.793+0000 7f8820800700 1 Processor -- start 2024-06-02T15:39:35.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.793+0000 7f8820800700 1 -- start start 2024-06-02T15:39:35.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f881819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:35.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8818076090 0x7f881819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:35.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 0x7f88181a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8818115970 con 0x7f8818075770 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88181157f0 con 0x7f8818076090 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f8820800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8818115af0 con 0x7f881810ff70 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f881819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8818076090 0x7f881819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f881819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53848/0 (socket says 172.21.15.90:53848) 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881de00700 1 -- 172.21.15.90:0/1439678229 learned_addr learned my addr 172.21.15.90:0/1439678229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881e800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 0x7f88181a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881e800700 1 -- 172.21.15.90:0/1439678229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8818076090 msgr2=0x7f881819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881e800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8818076090 0x7f881819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881e800700 1 -- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 msgr2=0x7f881819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.794+0000 7f881e800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f881819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f881e800700 1 -- 172.21.15.90:0/1439678229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88181a0e70 con 0x7f881810ff70 2024-06-02T15:39:35.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f881e800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 0x7f88181a0730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f881400b070 tx=0x7f8814004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814021070 con 0x7f881810ff70 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8814007e30 con 0x7f881810ff70 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88181a1100 con 0x7f881810ff70 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814004570 con 0x7f881810ff70 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.795+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a89e0 con 0x7f881810ff70 2024-06-02T15:39:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.796+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881806a230 con 0x7f881810ff70 2024-06-02T15:39:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.799+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8814004710 con 0x7f881810ff70 2024-06-02T15:39:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.800+0000 7f8812a00700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 0x7f88000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.801+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f881400f020 con 0x7f881810ff70 2024-06-02T15:39:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.801+0000 7f881de00700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 0x7f88000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.801+0000 7f881de00700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 0x7f88000b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f880c009680 tx=0x7f880c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:35.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:35.802+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f881402a020 con 0x7f881810ff70 2024-06-02T15:39:36.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.095+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f88181a8bb0 con 0x7f881810ff70 2024-06-02T15:39:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.145+0000 7f8812a00700 1 -- 172.21.15.90:0/1439678229 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f88140ab590 con 0x7f881810ff70 2024-06-02T15:39:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 msgr2=0x7f88000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 0x7f88000b4d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f880c009680 tx=0x7f880c009260 comp rx=0 tx=0).stop 2024-06-02T15:39:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 msgr2=0x7f88181a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 0x7f88181a0730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f881400b070 tx=0x7f8814004c80 comp rx=0 tx=0).stop 2024-06-02T15:39:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 shutdown_connections 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88000b2880 0x7f88000b4d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f881810ff70 0x7f88181a0730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818075770 0x7f881819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 --2- 172.21.15.90:0/1439678229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8818076090 0x7f881819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.148+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 >> 172.21.15.90:0/1439678229 conn(0x7f881807a6f0 msgr2=0x7f8818101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.149+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 shutdown_connections 2024-06-02T15:39:36.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:36.149+0000 7f8820800700 1 -- 172.21.15.90:0/1439678229 wait complete. 2024-06-02T15:39:37.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-02T15:39:37.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:36 smithi090 ceph-mon[31595]: pgmap v367: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-02T15:39:37.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:36 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1439678229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:36 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:36 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T15:39:37.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:36 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:36 smithi120 ceph-mon[36451]: pgmap v367: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-02T15:39:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:36 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1439678229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:36 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:36 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T15:39:37.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:36 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:36 smithi135 ceph-mon[36839]: pgmap v367: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-06-02T15:39:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:36 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1439678229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:36 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T15:39:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:36 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T15:39:37.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:36 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:37.571 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:38 smithi120 ceph-mon[36451]: pgmap v368: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-02T15:39:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:38 smithi135 ceph-mon[36839]: pgmap v368: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-02T15:39:39.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:38 smithi090 ceph-mon[31595]: pgmap v368: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-06-02T15:39:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 -- 172.21.15.90:0/1007440056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb88010eeb0 msgr2=0x7fb880116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1007440056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb88010eeb0 0x7fb880116a80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb878009f90 tx=0x7fb87800b330 comp rx=0 tx=0).stop 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 -- 172.21.15.90:0/1007440056 shutdown_connections 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1007440056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb88010eeb0 0x7fb880116a80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1007440056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb880109dd0 0x7fb88010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1007440056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb88010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 -- 172.21.15.90:0/1007440056 >> 172.21.15.90:0/1007440056 conn(0x7fb880076d10 msgr2=0x7fb880108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 -- 172.21.15.90:0/1007440056 shutdown_connections 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.942+0000 7fb885400700 1 -- 172.21.15.90:0/1007440056 wait complete. 2024-06-02T15:39:39.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.943+0000 7fb885400700 1 Processor -- start 2024-06-02T15:39:39.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.943+0000 7fb885400700 1 -- start start 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880109dd0 0x7fb8801a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 0x7fb8801a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb880119d90 con 0x7fb88010a780 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb880119c10 con 0x7fb880109dd0 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb885400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb880119f10 con 0x7fb88010eeb0 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb87de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb87de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53860/0 (socket says 172.21.15.90:53860) 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb87f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 0x7fb8801a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb87de00700 1 -- 172.21.15.90:0/1218711949 learned_addr learned my addr 172.21.15.90:0/1218711949 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.944+0000 7fb87e800700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880109dd0 0x7fb8801a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 -- 172.21.15.90:0/1218711949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 msgr2=0x7fb8801a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 0x7fb8801a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 -- 172.21.15.90:0/1218711949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880109dd0 msgr2=0x7fb8801a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880109dd0 0x7fb8801a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 -- 172.21.15.90:0/1218711949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8801a51e0 con 0x7fb88010a780 2024-06-02T15:39:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87f200700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 0x7fb8801a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb87de00700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e9f0 tx=0x7fb86c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00c9c0 con 0x7fb88010a780 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb86c00cb60 con 0x7fb88010a780 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8801a54d0 con 0x7fb88010a780 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.945+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c018680 con 0x7fb88010a780 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.946+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801acd50 con 0x7fb88010a780 2024-06-02T15:39:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.947+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88006a230 con 0x7fb88010a780 2024-06-02T15:39:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.950+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb86c010430 con 0x7fb88010a780 2024-06-02T15:39:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.951+0000 7fb877400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 0x7fb8640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:39.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.952+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb86c0e2700 con 0x7fb88010a780 2024-06-02T15:39:39.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.952+0000 7fb87e800700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 0x7fb8640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:39.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.953+0000 7fb87e800700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 0x7fb8640b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb870005950 tx=0x7fb87000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:39.956+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb86c0aa060 con 0x7fb88010a780 2024-06-02T15:39:40.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.239+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb88002e560 con 0x7fb88010a780 2024-06-02T15:39:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.937+0000 7fb877400700 1 -- 172.21.15.90:0/1218711949 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fb86c0b20e0 con 0x7fb88010a780 2024-06-02T15:39:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 msgr2=0x7fb8640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:40.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 0x7fb8640b4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb870005950 tx=0x7fb87000b410 comp rx=0 tx=0).stop 2024-06-02T15:39:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 msgr2=0x7fb8801a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00e9f0 tx=0x7fb86c00ed00 comp rx=0 tx=0).stop 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 shutdown_connections 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb8640b2880 0x7fb8640b4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb88010eeb0 0x7fb8801a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb880109dd0 0x7fb8801a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 --2- 172.21.15.90:0/1218711949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb88010a780 0x7fb8801a0760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.942+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 >> 172.21.15.90:0/1218711949 conn(0x7fb880076d10 msgr2=0x7fb8800788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.943+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 shutdown_connections 2024-06-02T15:39:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:40.943+0000 7fb885400700 1 -- 172.21.15.90:0/1218711949 wait complete. 2024-06-02T15:39:41.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:40 smithi090 ceph-mon[31595]: pgmap v369: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-02T15:39:41.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T15:39:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:40 smithi120 ceph-mon[36451]: pgmap v369: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-02T15:39:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T15:39:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:40 smithi135 ceph-mon[36839]: pgmap v369: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-06-02T15:39:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T15:39:41.837 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-06-02T15:39:41.837 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-02T15:39:42.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:41 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T15:39:42.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:41 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:42.272 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:42.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:41 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T15:39:42.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:41 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:42.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:41 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1218711949' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T15:39:42.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:41 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:39:43.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:42 smithi090 ceph-mon[31595]: pgmap v370: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-02T15:39:43.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:42 smithi120 ceph-mon[36451]: pgmap v370: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-02T15:39:43.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:42 smithi135 ceph-mon[36839]: pgmap v370: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.422+0000 7f5111400700 1 -- 172.21.15.90:0/2642994781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1076b0 msgr2=0x7f510c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.422+0000 7f5111400700 1 --2- 172.21.15.90:0/2642994781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1076b0 0x7f510c10f280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f510400b0d0 tx=0x7f510400b3e0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 -- 172.21.15.90:0/2642994781 shutdown_connections 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 --2- 172.21.15.90:0/2642994781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1076b0 0x7f510c10f280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 --2- 172.21.15.90:0/2642994781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1025d0 0x7f510c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 --2- 172.21.15.90:0/2642994781 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 -- 172.21.15.90:0/2642994781 >> 172.21.15.90:0/2642994781 conn(0x7f510c0fde60 msgr2=0x7f510c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 -- 172.21.15.90:0/2642994781 shutdown_connections 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.423+0000 7f5111400700 1 -- 172.21.15.90:0/2642994781 wait complete. 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 Processor -- start 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 -- start start 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1025d0 0x7f510c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 0x7f510c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1125c0 con 0x7f510c1025d0 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f510c112440 con 0x7f510c102f80 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f5111400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f510c112740 con 0x7f510c1076b0 2024-06-02T15:39:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f510be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1025d0 0x7f510c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.424+0000 7f510b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35536/0 (socket says 172.21.15.90:35536) 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 -- 172.21.15.90:0/74893487 learned_addr learned my addr 172.21.15.90:0/74893487 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5110a00700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 0x7f510c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 -- 172.21.15.90:0/74893487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 msgr2=0x7f510c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 0x7f510c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 -- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1025d0 msgr2=0x7f510c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1025d0 0x7f510c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 -- 172.21.15.90:0/74893487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a0d30 con 0x7f510c102f80 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5110a00700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 0x7f510c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f510b400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50f800e970 tx=0x7f50f800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f800cbd0 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a1020 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f800cd70 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.425+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8005680 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.426+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a88a0 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.427+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c06a230 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.429+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f50f8004750 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.430+0000 7f5108c00700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 0x7f50f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.431+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f50f80e29a0 con 0x7f510c102f80 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.431+0000 7f510be00700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 0x7f50f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.431+0000 7f510be00700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 0x7f50f00b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009c60 tx=0x7f50fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.433+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50f8054df0 con 0x7f510c102f80 2024-06-02T15:39:44.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.704+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f510c19d080 con 0x7f50f00b2880 2024-06-02T15:39:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.739+0000 7f5108c00700 1 -- 172.21.15.90:0/74893487 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f510c19d080 con 0x7f50f00b2880 2024-06-02T15:39:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 msgr2=0x7f50f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 0x7f50f00b4d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009c60 tx=0x7f50fc009260 comp rx=0 tx=0).stop 2024-06-02T15:39:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 msgr2=0x7f510c19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50f800e970 tx=0x7f50f800ec80 comp rx=0 tx=0).stop 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 shutdown_connections 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f50f00b2880 0x7f50f00b4d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f510c1076b0 0x7f510c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f510c1025d0 0x7f510c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 --2- 172.21.15.90:0/74893487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f510c102f80 0x7f510c19c260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 >> 172.21.15.90:0/74893487 conn(0x7f510c0fde60 msgr2=0x7f510c10ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 shutdown_connections 2024-06-02T15:39:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:44.742+0000 7f5111400700 1 -- 172.21.15.90:0/74893487 wait complete. 2024-06-02T15:39:44.873 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:39:44.712+0000 7eff20200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T15:39:44.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:39:44.712+0000 7eff20200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T15:39:44.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:39:44.712+0000 7eff20200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T15:39:44.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:39:44.712+0000 7eff20200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T15:39:44.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:39:44.712+0000 7eff20200700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T15:39:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:44 smithi120 ceph-mon[36451]: pgmap v371: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:39:45.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:44 smithi135 ceph-mon[36839]: pgmap v371: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:39:45.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:44 smithi090 ceph-mon[31595]: pgmap v371: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:39:45.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-02T15:39:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:46.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:45 smithi090 ceph-mon[31595]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:46.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:45 smithi120 ceph-mon[36451]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:46.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:45 smithi135 ceph-mon[36839]: from='client.24701 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:47.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:46 smithi120 ceph-mon[36451]: pgmap v372: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-02T15:39:47.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:46 smithi120 ceph-mon[36451]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-02T15:39:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:46 smithi135 ceph-mon[36839]: pgmap v372: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-02T15:39:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:46 smithi135 ceph-mon[36839]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-02T15:39:47.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:46 smithi090 ceph-mon[31595]: pgmap v372: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-06-02T15:39:47.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:46 smithi090 ceph-mon[31595]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 -- 172.21.15.90:0/1758018714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 msgr2=0x7f0d781141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/1758018714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781141d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d600098f0 tx=0x7f0d60009c00 comp rx=0 tx=0).stop 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 -- 172.21.15.90:0/1758018714 shutdown_connections 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/1758018714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78114710 0x7f0d78116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/1758018714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78075770 0x7f0d78075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/1758018714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781141d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 -- 172.21.15.90:0/1758018714 >> 172.21.15.90:0/1758018714 conn(0x7f0d7807a6a0 msgr2=0x7f0d7807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 -- 172.21.15.90:0/1758018714 shutdown_connections 2024-06-02T15:39:48.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.225+0000 7f0d7ca00700 1 -- 172.21.15.90:0/1758018714 wait complete. 2024-06-02T15:39:48.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.226+0000 7f0d7ca00700 1 Processor -- start 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 -- start start 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78114710 0x7f0d781a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78119d20 con 0x7f0d78114710 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78119ba0 con 0x7f0d78075770 2024-06-02T15:39:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.227+0000 7f0d7ca00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78119ea0 con 0x7f0d78076090 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35554/0 (socket says 172.21.15.90:35554) 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 -- 172.21.15.90:0/2707579062 learned_addr learned my addr 172.21.15.90:0/2707579062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d6d400700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d76800700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78114710 0x7f0d781a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 -- 172.21.15.90:0/2707579062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 msgr2=0x7f0d781a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 -- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78114710 msgr2=0x7f0d781a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78114710 0x7f0d781a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.228+0000 7f0d75e00700 1 -- 172.21.15.90:0/2707579062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781a5260 con 0x7f0d78075770 2024-06-02T15:39:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d75e00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800ec10 tx=0x7f0d6800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800cc90 con 0x7f0d78075770 2024-06-02T15:39:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781a5550 con 0x7f0d78075770 2024-06-02T15:39:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6800ce30 con 0x7f0d78075770 2024-06-02T15:39:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68010430 con 0x7f0d78075770 2024-06-02T15:39:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.229+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781acdd0 con 0x7f0d78075770 2024-06-02T15:39:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.230+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7806a230 con 0x7f0d78075770 2024-06-02T15:39:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.232+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0d68004000 con 0x7f0d78075770 2024-06-02T15:39:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.234+0000 7f0d6f400700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 0x7f0d5c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.234+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d680e2a00 con 0x7f0d78075770 2024-06-02T15:39:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.234+0000 7f0d6d400700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 0x7f0d5c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.235+0000 7f0d6d400700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 0x7f0d5c0b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d60019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.238+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d680aa200 con 0x7f0d78075770 2024-06-02T15:39:48.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.511+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0d781108c0 con 0x7f0d5c0b2910 2024-06-02T15:39:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.554+0000 7f0d6f400700 1 -- 172.21.15.90:0/2707579062 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d781108c0 con 0x7f0d5c0b2910 2024-06-02T15:39:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 msgr2=0x7f0d5c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 0x7f0d5c0b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d60019040 comp rx=0 tx=0).stop 2024-06-02T15:39:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 msgr2=0x7f0d781a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800ec10 tx=0x7f0d6800ef20 comp rx=0 tx=0).stop 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 shutdown_connections 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0d5c0b2910 0x7f0d5c0b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d78114710 0x7f0d781a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0d78075770 0x7f0d781a0250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 --2- 172.21.15.90:0/2707579062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d78076090 0x7f0d781a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 >> 172.21.15.90:0/2707579062 conn(0x7f0d7807a6a0 msgr2=0x7f0d781101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 shutdown_connections 2024-06-02T15:39:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:48.558+0000 7f0d7ca00700 1 -- 172.21.15.90:0/2707579062 wait complete. 2024-06-02T15:39:49.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:48 smithi120 ceph-mon[36451]: pgmap v373: 129 pgs: 129 active+clean; 590 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-06-02T15:39:49.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:48 smithi135 ceph-mon[36839]: pgmap v373: 129 pgs: 129 active+clean; 590 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-06-02T15:39:49.338 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:48 smithi090 ceph-mon[31595]: pgmap v373: 129 pgs: 129 active+clean; 590 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-06-02T15:39:49.415 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-02T15:39:49.863 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:50.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:49 smithi090 ceph-mon[31595]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:50.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:49 smithi120 ceph-mon[36451]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:50.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:49 smithi135 ceph-mon[36839]: from='client.24713 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:51.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:50 smithi090 ceph-mon[31595]: pgmap v374: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-02T15:39:51.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:50 smithi120 ceph-mon[36451]: pgmap v374: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-02T15:39:51.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:50 smithi135 ceph-mon[36839]: pgmap v374: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 8 op/s 2024-06-02T15:39:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 -- 172.21.15.90:0/666880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241037f0 msgr2=0x7f2924103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 --2- 172.21.15.90:0/666880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241037f0 0x7f2924103bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2914009920 tx=0x7f2914009c30 comp rx=0 tx=0).stop 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 -- 172.21.15.90:0/666880561 shutdown_connections 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 --2- 172.21.15.90:0/666880561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 0x7f2924110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 --2- 172.21.15.90:0/666880561 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241037f0 0x7f2924103bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 --2- 172.21.15.90:0/666880561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241041a0 0x7f29241081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.071+0000 7f292a400700 1 -- 172.21.15.90:0/666880561 >> 172.21.15.90:0/666880561 conn(0x7f292407a6a0 msgr2=0x7f292407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.072+0000 7f292a400700 1 -- 172.21.15.90:0/666880561 shutdown_connections 2024-06-02T15:39:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.072+0000 7f292a400700 1 -- 172.21.15.90:0/666880561 wait complete. 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.073+0000 7f292a400700 1 Processor -- start 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.073+0000 7f292a400700 1 -- start start 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 0x7f292419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 0x7f29241a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113710 con 0x7f29241037f0 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113590 con 0x7f29241041a0 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f292a400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2924113890 con 0x7f29241088f0 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f2923400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f2923400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40844/0 (socket says 172.21.15.90:40844) 2024-06-02T15:39:52.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f2923400700 1 -- 172.21.15.90:0/1248836908 learned_addr learned my addr 172.21.15.90:0/1248836908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f2922a00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 0x7f292419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.074+0000 7f2923e00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 0x7f29241a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 -- 172.21.15.90:0/1248836908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 msgr2=0x7f29241a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 0x7f29241a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 -- 172.21.15.90:0/1248836908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 msgr2=0x7f292419c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 0x7f292419c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 -- 172.21.15.90:0/1248836908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241a0ed0 con 0x7f29241037f0 2024-06-02T15:39:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2922a00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 0x7f292419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f2923400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f291400c010 tx=0x7f291400fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914027070 con 0x7f29241037f0 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.075+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29241a1160 con 0x7f29241037f0 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.076+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291400b560 con 0x7f29241037f0 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.076+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914004930 con 0x7f29241037f0 2024-06-02T15:39:52.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.076+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a8a40 con 0x7f29241037f0 2024-06-02T15:39:52.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.077+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292406a230 con 0x7f29241037f0 2024-06-02T15:39:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.079+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f291402a070 con 0x7f29241037f0 2024-06-02T15:39:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.080+0000 7f291be00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 0x7f29080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.081+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f29140e3120 con 0x7f29241037f0 2024-06-02T15:39:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.081+0000 7f2922a00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 0x7f29080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.081+0000 7f2922a00700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 0x7f29080b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2924103380 tx=0x7f2910009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:52.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.085+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f291405fdf0 con 0x7f29241037f0 2024-06-02T15:39:52.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.355+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f292410f7d0 con 0x7f29080b2910 2024-06-02T15:39:52.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.364+0000 7f291be00700 1 -- 172.21.15.90:0/1248836908 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f292410f7d0 con 0x7f29080b2910 2024-06-02T15:39:52.362 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_0/a56b163d-040c-48ac-aa1f-2c4e36e229e7 2024-06-02T15:39:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 msgr2=0x7f29080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 0x7f29080b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2924103380 tx=0x7f2910009310 comp rx=0 tx=0).stop 2024-06-02T15:39:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 msgr2=0x7f292419bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f291400c010 tx=0x7f291400fa90 comp rx=0 tx=0).stop 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 shutdown_connections 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f29080b2910 0x7f29080b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29241088f0 0x7f29241a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29241037f0 0x7f292419bec0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 --2- 172.21.15.90:0/1248836908 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29241041a0 0x7f292419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 >> 172.21.15.90:0/1248836908 conn(0x7f292407a6a0 msgr2=0x7f2924101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 shutdown_connections 2024-06-02T15:39:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:52.368+0000 7f292a400700 1 -- 172.21.15.90:0/1248836908 wait complete. 2024-06-02T15:39:53.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:52 smithi090 ceph-mon[31595]: pgmap v375: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:39:53.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:52 smithi090 ceph-mon[31595]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:53.193 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-02T15:39:53.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:52 smithi120 ceph-mon[36451]: pgmap v375: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:39:53.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:52 smithi120 ceph-mon[36451]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:53.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:52 smithi135 ceph-mon[36839]: pgmap v375: 129 pgs: 129 active+clean; 591 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:39:53.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:52 smithi135 ceph-mon[36839]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:53.645 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:55.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:54 smithi120 ceph-mon[36451]: pgmap v376: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:55.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:54 smithi090 ceph-mon[31595]: pgmap v376: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:55.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:54 smithi135 ceph-mon[36839]: pgmap v376: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:55.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 -- 172.21.15.90:0/681325178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01041a0 msgr2=0x7f63c01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:55.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 --2- 172.21.15.90:0/681325178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01041a0 0x7f63c01081c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63a80098f0 tx=0x7f63a8009c00 comp rx=0 tx=0).stop 2024-06-02T15:39:55.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 -- 172.21.15.90:0/681325178 shutdown_connections 2024-06-02T15:39:55.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 --2- 172.21.15.90:0/681325178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01088f0 0x7f63c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:55.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 --2- 172.21.15.90:0/681325178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01037f0 0x7f63c0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:55.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 --2- 172.21.15.90:0/681325178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01041a0 0x7f63c01081c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:55.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 -- 172.21.15.90:0/681325178 >> 172.21.15.90:0/681325178 conn(0x7f63c007a6a0 msgr2=0x7f63c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:55.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 -- 172.21.15.90:0/681325178 shutdown_connections 2024-06-02T15:39:55.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.795+0000 7f63c7c00700 1 -- 172.21.15.90:0/681325178 wait complete. 2024-06-02T15:39:55.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.797+0000 7f63c7c00700 1 Processor -- start 2024-06-02T15:39:55.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.797+0000 7f63c7c00700 1 -- start start 2024-06-02T15:39:55.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.797+0000 7f63c7c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 0x7f63c019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.797+0000 7f63c7c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.797+0000 7f63c7c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01088f0 0x7f63c0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c7c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01136b0 con 0x7f63c01041a0 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c7c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0113530 con 0x7f63c01037f0 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c7c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0113830 con 0x7f63c01088f0 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 0x7f63c019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40872/0 (socket says 172.21.15.90:40872) 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 -- 172.21.15.90:0/661851319 learned_addr learned my addr 172.21.15.90:0/661851319 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 0x7f63c019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:45454/0 (socket says 172.21.15.90:45454) 2024-06-02T15:39:55.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63c5c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01088f0 0x7f63c0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:55.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 -- 172.21.15.90:0/661851319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01088f0 msgr2=0x7f63c0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:55.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01088f0 0x7f63c0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:55.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 -- 172.21.15.90:0/661851319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 msgr2=0x7f63c019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 0x7f63c019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.798+0000 7f63b7e00700 1 -- 172.21.15.90:0/661851319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c019a170 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63b7e00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009230 tx=0x7f63a800f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8027070 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a80055f0 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c019a400 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8004930 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.799+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c0075770 con 0x7f63c01041a0 2024-06-02T15:39:55.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.800+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0104c80 con 0x7f63c01041a0 2024-06-02T15:39:55.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.803+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f63a802a070 con 0x7f63c01041a0 2024-06-02T15:39:55.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.804+0000 7f63be000700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 0x7f63a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:55.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.804+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f63a80e30a0 con 0x7f63c01041a0 2024-06-02T15:39:55.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.804+0000 7f63c5200700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 0x7f63a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:55.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.805+0000 7f63c5200700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 0x7f63a40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63c019ab50 tx=0x7f63b0012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:55.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:55.809+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63a8013030 con 0x7f63c01041a0 2024-06-02T15:39:56.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.079+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f63c019af40 con 0x7f63a40b2910 2024-06-02T15:39:56.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.116+0000 7f63be000700 1 -- 172.21.15.90:0/661851319 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63c019af40 con 0x7f63a40b2910 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 msgr2=0x7f63a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 0x7f63a40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f63c019ab50 tx=0x7f63b0012040 comp rx=0 tx=0).stop 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 msgr2=0x7f63c019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009230 tx=0x7f63a800f9b0 comp rx=0 tx=0).stop 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 shutdown_connections 2024-06-02T15:39:56.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f63a40b2910 0x7f63a40b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c01088f0 0x7f63c0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f63c01037f0 0x7f63c019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 --2- 172.21.15.90:0/661851319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63c01041a0 0x7f63c019fd90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.119+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 >> 172.21.15.90:0/661851319 conn(0x7f63c007a6a0 msgr2=0x7f63c01018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.120+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 shutdown_connections 2024-06-02T15:39:56.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:56.120+0000 7f63c7c00700 1 -- 172.21.15.90:0/661851319 wait complete. 2024-06-02T15:39:56.943 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-02T15:39:57.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:57 smithi120 ceph-mon[36451]: pgmap v377: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:57.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:57 smithi120 ceph-mon[36451]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:57.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:56 smithi135 ceph-mon[36839]: pgmap v377: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:57.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:56 smithi135 ceph-mon[36839]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:57.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:56 smithi090 ceph-mon[31595]: pgmap v377: 129 pgs: 129 active+clean; 611 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2024-06-02T15:39:57.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:56 smithi090 ceph-mon[31595]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:39:57.398 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:39:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:59 smithi090 ceph-mon[31595]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-02T15:39:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-06-02T15:39:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-02T15:39:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:39:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-06-02T15:39:59.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:59 smithi120 ceph-mon[36451]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-02T15:39:59.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-06-02T15:39:59.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-02T15:39:59.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:39:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-06-02T15:39:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:59 smithi135 ceph-mon[36839]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-06-02T15:39:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-06-02T15:39:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-06-02T15:39:59.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:39:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]: dispatch 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 -- 172.21.15.90:0/4166619470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c104fa0 msgr2=0x7ff37c10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 --2- 172.21.15.90:0/4166619470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c104fa0 0x7ff37c10d490 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff368009990 tx=0x7ff368009ca0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 -- 172.21.15.90:0/4166619470 shutdown_connections 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 --2- 172.21.15.90:0/4166619470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c10dab0 0x7ff37c10ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 --2- 172.21.15.90:0/4166619470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104680 0x7ff37c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 --2- 172.21.15.90:0/4166619470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c104fa0 0x7ff37c10d490 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 -- 172.21.15.90:0/4166619470 >> 172.21.15.90:0/4166619470 conn(0x7ff37c0fded0 msgr2=0x7ff37c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 -- 172.21.15.90:0/4166619470 shutdown_connections 2024-06-02T15:39:59.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.699+0000 7ff383000700 1 -- 172.21.15.90:0/4166619470 wait complete. 2024-06-02T15:39:59.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.700+0000 7ff383000700 1 Processor -- start 2024-06-02T15:39:59.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 -- start start 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c104680 0x7ff37c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 0x7ff37c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c113700 con 0x7ff37c10dab0 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c113580 con 0x7ff37c104fa0 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff383000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c113880 con 0x7ff37c104680 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.701+0000 7ff382600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40888/0 (socket says 172.21.15.90:40888) 2024-06-02T15:39:59.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 -- 172.21.15.90:0/1727030679 learned_addr learned my addr 172.21.15.90:0/1727030679 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:39:59.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff381200700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 0x7ff37c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:59.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff381c00700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c104680 0x7ff37c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:59.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 -- 172.21.15.90:0/1727030679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c104680 msgr2=0x7ff37c19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:59.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c104680 0x7ff37c19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 -- 172.21.15.90:0/1727030679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 msgr2=0x7ff37c19c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 0x7ff37c19c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 -- 172.21.15.90:0/1727030679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c1a0e40 con 0x7ff37c10dab0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff381200700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 0x7ff37c19c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.702+0000 7ff382600700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff37400ef30 tx=0x7ff37400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.703+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3740092e0 con 0x7ff37c10dab0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.703+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374004bb0 con 0x7ff37c10dab0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.703+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004020 con 0x7ff37c10dab0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.703+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c1a10d0 con 0x7ff37c10dab0 2024-06-02T15:39:59.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.703+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c1a89b0 con 0x7ff37c10dab0 2024-06-02T15:39:59.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.704+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37c06a230 con 0x7ff37c10dab0 2024-06-02T15:39:59.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.706+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff37401b030 con 0x7ff37c10dab0 2024-06-02T15:39:59.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.709+0000 7ff37a000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 0x7ff3640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:39:59.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.709+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff3740e21e0 con 0x7ff37c10dab0 2024-06-02T15:39:59.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.709+0000 7ff381c00700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 0x7ff3640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:39:59.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.710+0000 7ff381c00700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 0x7ff3640b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff370009c60 tx=0x7ff370009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:39:59.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:39:59.714+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff374022020 con 0x7ff37c10dab0 2024-06-02T15:40:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.028+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff37c103ab0 con 0x7ff3640b2910 2024-06-02T15:40:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.038+0000 7ff37a000700 1 -- 172.21.15.90:0/1727030679 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff37c103ab0 con 0x7ff3640b2910 2024-06-02T15:40:00.036 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_1/6c423124-5043-4aa3-b141-d9bef9e296fb 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 msgr2=0x7ff3640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 0x7ff3640b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff370009c60 tx=0x7ff370009260 comp rx=0 tx=0).stop 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 msgr2=0x7ff37c1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff37400ef30 tx=0x7ff37400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 shutdown_connections 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff3640b2910 0x7ff3640b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff37c10dab0 0x7ff37c1a0700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff37c104680 0x7ff37c19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 --2- 172.21.15.90:0/1727030679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff37c104fa0 0x7ff37c19c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.042+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 >> 172.21.15.90:0/1727030679 conn(0x7ff37c0fded0 msgr2=0x7ff37c10edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.043+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 shutdown_connections 2024-06-02T15:40:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:00.043+0000 7ff383000700 1 -- 172.21.15.90:0/1727030679 wait complete. 2024-06-02T15:40:00.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-06-02T15:40:00.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-02T15:40:00.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:00 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-06-02T15:40:00.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:00 smithi120 ceph-mon[36451]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T15:40:00.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:00 smithi120 ceph-mon[36451]: overall HEALTH_OK 2024-06-02T15:40:00.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-06-02T15:40:00.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-02T15:40:00.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:00 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-06-02T15:40:00.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:00 smithi135 ceph-mon[36839]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T15:40:00.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:00 smithi135 ceph-mon[36839]: overall HEALTH_OK 2024-06-02T15:40:00.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-06-02T15:40:00.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-06-02T15:40:00.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:00 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 10]}]': finished 2024-06-02T15:40:00.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:00 smithi090 ceph-mon[31595]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T15:40:00.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:00 smithi090 ceph-mon[31595]: overall HEALTH_OK 2024-06-02T15:40:00.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:40:01.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:01 smithi120 ceph-mon[36451]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-06-02T15:40:01.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:01 smithi120 ceph-mon[36451]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:40:01.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:01 smithi120 ceph-mon[36451]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T15:40:01.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:01 smithi135 ceph-mon[36839]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-06-02T15:40:01.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:01 smithi135 ceph-mon[36839]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:40:01.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:01 smithi135 ceph-mon[36839]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T15:40:01.319 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:01 smithi090 ceph-mon[31595]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.3 KiB/s wr, 8 op/s 2024-06-02T15:40:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:01 smithi090 ceph-mon[31595]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:40:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:01 smithi090 ceph-mon[31595]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T15:40:02.338 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:02 smithi090 ceph-mon[31595]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-02T15:40:02.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:02 smithi120 ceph-mon[36451]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-02T15:40:02.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:02 smithi135 ceph-mon[36839]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-02T15:40:03.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:03 smithi090 ceph-mon[31595]: pgmap v382: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 -- 172.21.15.90:0/3549074506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 msgr2=0x7fc780112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 --2- 172.21.15.90:0/3549074506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc780112520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc778009f90 tx=0x7fc77800b330 comp rx=0 tx=0).stop 2024-06-02T15:40:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 -- 172.21.15.90:0/3549074506 shutdown_connections 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 --2- 172.21.15.90:0/3549074506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc780112520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 --2- 172.21.15.90:0/3549074506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc78010b650 0x7fc78010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 --2- 172.21.15.90:0/3549074506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc780105650 0x7fc780105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 -- 172.21.15.90:0/3549074506 >> 172.21.15.90:0/3549074506 conn(0x7fc780101180 msgr2=0x7fc7801035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 -- 172.21.15.90:0/3549074506 shutdown_connections 2024-06-02T15:40:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.453+0000 7fc786400700 1 -- 172.21.15.90:0/3549074506 wait complete. 2024-06-02T15:40:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.454+0000 7fc786400700 1 Processor -- start 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 -- start start 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 0x7fc7801a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc7801a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 0x7fc78019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc780115880 con 0x7fc780105650 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc780115700 con 0x7fc780106010 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc786400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc780115a00 con 0x7fc78010b650 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.455+0000 7fc785000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 0x7fc7801a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 0x7fc78019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 0x7fc7801a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49162/0 (socket says 172.21.15.90:49162) 2024-06-02T15:40:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc77fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc7801a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785000700 1 -- 172.21.15.90:0/716169532 learned_addr learned my addr 172.21.15.90:0/716169532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 -- 172.21.15.90:0/716169532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 msgr2=0x7fc7801a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc7801a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 -- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 msgr2=0x7fc7801a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 0x7fc7801a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 -- 172.21.15.90:0/716169532 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78019c260 con 0x7fc78010b650 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc77fe00700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc7801a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.456+0000 7fc785a00700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 0x7fc78019bb00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b6c0 tx=0x7fc77800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:03.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.457+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778021070 con 0x7fc78010b650 2024-06-02T15:40:03.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.457+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc778007d60 con 0x7fc78010b650 2024-06-02T15:40:03.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.457+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778005040 con 0x7fc78010b650 2024-06-02T15:40:03.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.457+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78019c4f0 con 0x7fc78010b650 2024-06-02T15:40:03.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.457+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7801a8810 con 0x7fc78010b650 2024-06-02T15:40:03.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.458+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc780106d90 con 0x7fc78010b650 2024-06-02T15:40:03.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.460+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc778024070 con 0x7fc78010b650 2024-06-02T15:40:03.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.462+0000 7fc77d600700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 0x7fc7680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:03.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.462+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc77800f020 con 0x7fc78010b650 2024-06-02T15:40:03.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.462+0000 7fc785000700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 0x7fc7680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:03.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.462+0000 7fc785000700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 0x7fc7680b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc774009750 tx=0x7fc774009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:03.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.464+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc77802b020 con 0x7fc78010b650 2024-06-02T15:40:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:03 smithi120 ceph-mon[36451]: pgmap v382: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:03 smithi135 ceph-mon[36839]: pgmap v382: 129 pgs: 1 peering, 128 active+clean; 635 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:03.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.746+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc78002e560 con 0x7fc78010b650 2024-06-02T15:40:03.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.747+0000 7fc77d600700 1 -- 172.21.15.90:0/716169532 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7fc778007420 con 0x7fc78010b650 2024-06-02T15:40:03.745 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:03.746 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:40:03.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.750+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 msgr2=0x7fc7680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:03.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.750+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 0x7fc7680b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc774009750 tx=0x7fc774009310 comp rx=0 tx=0).stop 2024-06-02T15:40:03.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.750+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 msgr2=0x7fc78019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:03.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.750+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 0x7fc78019bb00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b6c0 tx=0x7fc77800cc30 comp rx=0 tx=0).stop 2024-06-02T15:40:03.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 shutdown_connections 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc7680b2910 0x7fc7680b4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc78010b650 0x7fc78019bb00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc780105650 0x7fc7801a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 --2- 172.21.15.90:0/716169532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc780106010 0x7fc7801a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 >> 172.21.15.90:0/716169532 conn(0x7fc780101180 msgr2=0x7fc780101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 shutdown_connections 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:03.751+0000 7fc786400700 1 -- 172.21.15.90:0/716169532 wait complete. 2024-06-02T15:40:03.749 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:40:04.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:04 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/716169532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:40:04.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:04 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/716169532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:40:04.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:04 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/716169532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:40:04.594 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:40:05.042 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:05.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:05 smithi090 ceph-mon[31595]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:05 smithi120 ceph-mon[36451]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:05 smithi135 ceph-mon[36839]: pgmap v383: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 8 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:06.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:06 smithi090 ceph-mon[31595]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-02T15:40:06.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:06 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:40:06.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:06 smithi120 ceph-mon[36451]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-02T15:40:06.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:06 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:40:06.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:06 smithi135 ceph-mon[36839]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-02T15:40:06.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:06 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:40:07.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.299+0000 7f44d0200700 1 -- 172.21.15.90:0/1295135230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c81095b0 msgr2=0x7f44c8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.299+0000 7f44d0200700 1 --2- 172.21.15.90:0/1295135230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c81095b0 0x7f44c8109990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009990 tx=0x7f44bc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 -- 172.21.15.90:0/1295135230 shutdown_connections 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 --2- 172.21.15.90:0/1295135230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c806bbe0 0x7f44c81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 --2- 172.21.15.90:0/1295135230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c81095b0 0x7f44c8109990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 --2- 172.21.15.90:0/1295135230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806b220 0x7f44c806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 -- 172.21.15.90:0/1295135230 >> 172.21.15.90:0/1295135230 conn(0x7f44c80793f0 msgr2=0x7f44c8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 -- 172.21.15.90:0/1295135230 shutdown_connections 2024-06-02T15:40:07.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.300+0000 7f44d0200700 1 -- 172.21.15.90:0/1295135230 wait complete. 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.301+0000 7f44d0200700 1 Processor -- start 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.301+0000 7f44d0200700 1 -- start start 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 0x7f44c8102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 0x7f44c8102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c81095b0 0x7f44c8102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8113720 con 0x7f44c806b220 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81135a0 con 0x7f44c806bbe0 2024-06-02T15:40:07.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44d0200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44c81138a0 con 0x7f44c81095b0 2024-06-02T15:40:07.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cd800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 0x7f44c8102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:07.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cce00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 0x7f44c8102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cd800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 0x7f44c8102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49182/0 (socket says 172.21.15.90:49182) 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cd800700 1 -- 172.21.15.90:0/2612750014 learned_addr learned my addr 172.21.15.90:0/2612750014 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44ce200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c81095b0 0x7f44c8102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cce00700 1 -- 172.21.15.90:0/2612750014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c81095b0 msgr2=0x7f44c8102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.302+0000 7f44cce00700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c81095b0 0x7f44c8102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44cce00700 1 -- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 msgr2=0x7f44c8102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44cce00700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 0x7f44c8102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44cce00700 1 -- 172.21.15.90:0/2612750014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c81a80c0 con 0x7f44c806bbe0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44cce00700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 0x7f44c8102a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400cb70 con 0x7f44c806bbe0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c81a83b0 con 0x7f44c806bbe0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b400ee90 con 0x7f44c806bbe0 2024-06-02T15:40:07.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4003d40 con 0x7f44c806bbe0 2024-06-02T15:40:07.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.303+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c81a8940 con 0x7f44c806bbe0 2024-06-02T15:40:07.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.305+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c810ba90 con 0x7f44c806bbe0 2024-06-02T15:40:07.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.307+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f44b4003ee0 con 0x7f44c806bbe0 2024-06-02T15:40:07.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.308+0000 7f44c6000700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 0x7f44b00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:07.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.308+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f44b40e2890 con 0x7f44c806bbe0 2024-06-02T15:40:07.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.308+0000 7f44cd800700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 0x7f44b00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:07.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.309+0000 7f44cd800700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 0x7f44b00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:07.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.312+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44b40aa040 con 0x7f44c806bbe0 2024-06-02T15:40:07.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.547+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44c8103a40 con 0x7f44c806bbe0 2024-06-02T15:40:07.545 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:07 smithi090 ceph-mon[31595]: pgmap v384: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.549+0000 7f44c6000700 1 -- 172.21.15.90:0/2612750014 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7f44b40a9940 con 0x7f44c806bbe0 2024-06-02T15:40:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:07.547 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:40:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 msgr2=0x7f44b00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 0x7f44b00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc00b280 comp rx=0 tx=0).stop 2024-06-02T15:40:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 msgr2=0x7f44c8102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 0x7f44c8102a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400ed00 comp rx=0 tx=0).stop 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 shutdown_connections 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44b00b2910 0x7f44b00b4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44c81095b0 0x7f44c8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44c806b220 0x7f44c8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 --2- 172.21.15.90:0/2612750014 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44c806bbe0 0x7f44c8102a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 >> 172.21.15.90:0/2612750014 conn(0x7f44c80793f0 msgr2=0x7f44c81010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 shutdown_connections 2024-06-02T15:40:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:07.553+0000 7f44d0200700 1 -- 172.21.15.90:0/2612750014 wait complete. 2024-06-02T15:40:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:07 smithi120 ceph-mon[36451]: pgmap v384: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:07 smithi135 ceph-mon[36839]: pgmap v384: 129 pgs: 129 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-06-02T15:40:08.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:08 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2612750014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:08.547 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-02T15:40:08.547 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T15:40:08.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:08 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2612750014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:08.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:08 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2612750014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:08.989 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:09.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:09 smithi090 ceph-mon[31595]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s; 13 B/s, 0 objects/s recovering 2024-06-02T15:40:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:09 smithi120 ceph-mon[36451]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s; 13 B/s, 0 objects/s recovering 2024-06-02T15:40:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:09 smithi135 ceph-mon[36839]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 8 op/s; 13 B/s, 0 objects/s recovering 2024-06-02T15:40:11.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 -- 172.21.15.90:0/3106320144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 msgr2=0x7f6194109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 --2- 172.21.15.90:0/3106320144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194109990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6184009990 tx=0x7f6184009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 -- 172.21.15.90:0/3106320144 shutdown_connections 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 --2- 172.21.15.90:0/3106320144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 0x7f61941103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 --2- 172.21.15.90:0/3106320144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194109990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 --2- 172.21.15.90:0/3106320144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 -- 172.21.15.90:0/3106320144 >> 172.21.15.90:0/3106320144 conn(0x7f61940793f0 msgr2=0x7f6194079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 -- 172.21.15.90:0/3106320144 shutdown_connections 2024-06-02T15:40:11.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.187+0000 7f619b600700 1 -- 172.21.15.90:0/3106320144 wait complete. 2024-06-02T15:40:11.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.188+0000 7f619b600700 1 Processor -- start 2024-06-02T15:40:11.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.188+0000 7f619b600700 1 -- start start 2024-06-02T15:40:11.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 0x7f619419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6194113720 con 0x7f61941095b0 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61941135a0 con 0x7f619406bbe0 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f619b600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61941138a0 con 0x7f619406b220 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f618be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 0x7f619419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6199600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:43984/0 (socket says 172.21.15.90:43984) 2024-06-02T15:40:11.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 -- 172.21.15.90:0/2535701765 learned_addr learned my addr 172.21.15.90:0/2535701765 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:11.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 -- 172.21.15.90:0/2535701765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 msgr2=0x7f619419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:11.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 0x7f619419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 -- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 msgr2=0x7f6194199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.189+0000 7f6198c00700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f6198c00700 1 -- 172.21.15.90:0/2535701765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619419a1c0 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f6198c00700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6184005910 tx=0x7f618400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184027070 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6184022470 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619419a450 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184017630 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.190+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941a6770 con 0x7f619406b220 2024-06-02T15:40:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.191+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619410ba90 con 0x7f619406b220 2024-06-02T15:40:11.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.194+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f618402a070 con 0x7f619406b220 2024-06-02T15:40:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.195+0000 7f6192000700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 0x7f61780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:11.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.195+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f61840e38f0 con 0x7f619406b220 2024-06-02T15:40:11.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.196+0000 7f618be00700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 0x7f61780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:11.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.196+0000 7f618be00700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 0x7f61780b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6194079ad0 tx=0x7f617c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.197+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6184013070 con 0x7f619406b220 2024-06-02T15:40:11.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:11 smithi090 ceph-mon[31595]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-06-02T15:40:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.424+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f619419af90 con 0x7f61780b2910 2024-06-02T15:40:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.445+0000 7f6192000700 1 -- 172.21.15.90:0/2535701765 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7f619419af90 con 0x7f61780b2910 2024-06-02T15:40:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 msgr2=0x7f61780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 0x7f61780b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6194079ad0 tx=0x7f617c009260 comp rx=0 tx=0).stop 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 msgr2=0x7f619419f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6184005910 tx=0x7f618400b670 comp rx=0 tx=0).stop 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 shutdown_connections 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61780b2910 0x7f61780b4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61941095b0 0x7f6194199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f619406b220 0x7f619419f8a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 --2- 172.21.15.90:0/2535701765 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f619406bbe0 0x7f619419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:11.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 >> 172.21.15.90:0/2535701765 conn(0x7f61940793f0 msgr2=0x7f6194101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:11.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 shutdown_connections 2024-06-02T15:40:11.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:11.452+0000 7f619b600700 1 -- 172.21.15.90:0/2535701765 wait complete. 2024-06-02T15:40:11.454 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:40:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:11 smithi120 ceph-mon[36451]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-06-02T15:40:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:11 smithi135 ceph-mon[36839]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s; 11 B/s, 0 objects/s recovering 2024-06-02T15:40:12.297 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-06-02T15:40:09.732041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659054,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":301,"num_read_kb":249,"num_write":294,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375872,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565952,"statfs":{"total":1151940427776,"available":1151555534848,"internally_reserved":0,"allocated":14524416,"data_stored":8162482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":40,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031464+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:40:00.031464+0000","last_peered":"2024-06-02T15:40:00.031464+0000","last_clean":"2024-06-02T15:40:00.031464+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:40:00.031464+0000","last_undegraded":"2024-06-02T15:40:00.031464+0000","last_fullsized":"2024-06-02T15:40:00.031464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028321+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:40:00.028321+0000","last_peered":"2024-06-02T15:40:00.028321+0000","last_clean":"2024-06-02T15:40:00.028321+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:40:00.028321+0000","last_undegraded":"2024-06-02T15:40:00.028321+0000","last_fullsized":"2024-06-02T15:40:00.028321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026888+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:40:00.026888+0000","last_peered":"2024-06-02T15:40:00.026888+0000","last_clean":"2024-06-02T15:40:00.026888+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:40:00.026888+0000","last_undegraded":"2024-06-02T15:40:00.026888+0000","last_fullsized":"2024-06-02T15:40:00.026888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031981+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:40:00.031981+0000","last_peered":"2024-06-02T15:40:00.031981+0000","last_clean":"2024-06-02T15:40:00.031981+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:40:00.031981+0000","last_undegraded":"2024-06-02T15:40:00.031981+0000","last_fullsized":"2024-06-02T15:40:00.031981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032260+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:40:00.032260+0000","last_peered":"2024-06-02T15:40:00.032260+0000","last_clean":"2024-06-02T15:40:00.032260+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:40:00.032260+0000","last_undegraded":"2024-06-02T15:40:00.032260+0000","last_fullsized":"2024-06-02T15:40:00.032260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031691+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:40:00.031691+0000","last_peered":"2024-06-02T15:40:00.031691+0000","last_clean":"2024-06-02T15:40:00.031691+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:40:00.031691+0000","last_undegraded":"2024-06-02T15:40:00.031691+0000","last_fullsized":"2024-06-02T15:40:00.031691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026768+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:40:00.026768+0000","last_peered":"2024-06-02T15:40:00.026768+0000","last_clean":"2024-06-02T15:40:00.026768+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:40:00.026768+0000","last_undegraded":"2024-06-02T15:40:00.026768+0000","last_fullsized":"2024-06-02T15:40:00.026768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032189+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:40:00.032189+0000","last_peered":"2024-06-02T15:40:00.032189+0000","last_clean":"2024-06-02T15:40:00.032189+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:40:00.032189+0000","last_undegraded":"2024-06-02T15:40:00.032189+0000","last_fullsized":"2024-06-02T15:40:00.032189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028782+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:40:00.028782+0000","last_peered":"2024-06-02T15:40:00.028782+0000","last_clean":"2024-06-02T15:40:00.028782+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:40:00.028782+0000","last_undegraded":"2024-06-02T15:40:00.028782+0000","last_fullsized":"2024-06-02T15:40:00.028782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032137+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:40:00.032137+0000","last_peered":"2024-06-02T15:40:00.032137+0000","last_clean":"2024-06-02T15:40:00.032137+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:40:00.032137+0000","last_undegraded":"2024-06-02T15:40:00.032137+0000","last_fullsized":"2024-06-02T15:40:00.032137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027808+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:40:00.027808+0000","last_peered":"2024-06-02T15:40:00.027808+0000","last_clean":"2024-06-02T15:40:00.027808+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:40:00.027808+0000","last_undegraded":"2024-06-02T15:40:00.027808+0000","last_fullsized":"2024-06-02T15:40:00.027808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031351+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:40:00.031351+0000","last_peered":"2024-06-02T15:40:00.031351+0000","last_clean":"2024-06-02T15:40:00.031351+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:40:00.031351+0000","last_undegraded":"2024-06-02T15:40:00.031351+0000","last_fullsized":"2024-06-02T15:40:00.031351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034138+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:40:00.034138+0000","last_peered":"2024-06-02T15:40:00.034138+0000","last_clean":"2024-06-02T15:40:00.034138+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:40:00.034138+0000","last_undegraded":"2024-06-02T15:40:00.034138+0000","last_fullsized":"2024-06-02T15:40:00.034138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026409+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:40:00.026409+0000","last_peered":"2024-06-02T15:40:00.026409+0000","last_clean":"2024-06-02T15:40:00.026409+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:40:00.026409+0000","last_undegraded":"2024-06-02T15:40:00.026409+0000","last_fullsized":"2024-06-02T15:40:00.026409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026214+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:40:00.026214+0000","last_peered":"2024-06-02T15:40:00.026214+0000","last_clean":"2024-06-02T15:40:00.026214+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:40:00.026214+0000","last_undegraded":"2024-06-02T15:40:00.026214+0000","last_fullsized":"2024-06-02T15:40:00.026214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027558+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:40:00.027558+0000","last_peered":"2024-06-02T15:40:00.027558+0000","last_clean":"2024-06-02T15:40:00.027558+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:40:00.027558+0000","last_undegraded":"2024-06-02T15:40:00.027558+0000","last_fullsized":"2024-06-02T15:40:00.027558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031210+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:40:00.031210+0000","last_peered":"2024-06-02T15:40:00.031210+0000","last_clean":"2024-06-02T15:40:00.031210+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:40:00.031210+0000","last_undegraded":"2024-06-02T15:40:00.031210+0000","last_fullsized":"2024-06-02T15:40:00.031210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036187+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:40:00.036187+0000","last_peered":"2024-06-02T15:40:00.036187+0000","last_clean":"2024-06-02T15:40:00.036187+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:40:00.036187+0000","last_undegraded":"2024-06-02T15:40:00.036187+0000","last_fullsized":"2024-06-02T15:40:00.036187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031209+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:40:00.031209+0000","last_peered":"2024-06-02T15:40:00.031209+0000","last_clean":"2024-06-02T15:40:00.031209+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:40:00.031209+0000","last_undegraded":"2024-06-02T15:40:00.031209+0000","last_fullsized":"2024-06-02T15:40:00.031209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030505+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:00.030505+0000","last_peered":"2024-06-02T15:40:00.030505+0000","last_clean":"2024-06-02T15:40:00.030505+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:00.030505+0000","last_undegraded":"2024-06-02T15:40:00.030505+0000","last_fullsized":"2024-06-02T15:40:00.030505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036698+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:00.036698+0000","last_peered":"2024-06-02T15:40:00.036698+0000","last_clean":"2024-06-02T15:40:00.036698+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:00.036698+0000","last_undegraded":"2024-06-02T15:40:00.036698+0000","last_fullsized":"2024-06-02T15:40:00.036698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034575+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:40:00.034575+0000","last_peered":"2024-06-02T15:40:00.034575+0000","last_clean":"2024-06-02T15:40:00.034575+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:40:00.034575+0000","last_undegraded":"2024-06-02T15:40:00.034575+0000","last_fullsized":"2024-06-02T15:40:00.034575+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.041841+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:00.041841+0000","last_peered":"2024-06-02T15:40:00.041841+0000","last_clean":"2024-06-02T15:40:00.041841+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:00.041841+0000","last_undegraded":"2024-06-02T15:40:00.041841+0000","last_fullsized":"2024-06-02T15:40:00.041841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034771+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:00.034771+0000","last_peered":"2024-06-02T15:40:00.034771+0000","last_clean":"2024-06-02T15:40:00.034771+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:00.034771+0000","last_undegraded":"2024-06-02T15:40:00.034771+0000","last_fullsized":"2024-06-02T15:40:00.034771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029136+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:40:00.029136+0000","last_peered":"2024-06-02T15:40:00.029136+0000","last_clean":"2024-06-02T15:40:00.029136+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:40:00.029136+0000","last_undegraded":"2024-06-02T15:40:00.029136+0000","last_fullsized":"2024-06-02T15:40:00.029136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031380+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031380+0000","last_peered":"2024-06-02T15:40:00.031380+0000","last_clean":"2024-06-02T15:40:00.031380+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031380+0000","last_undegraded":"2024-06-02T15:40:00.031380+0000","last_fullsized":"2024-06-02T15:40:00.031380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032541+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:00.032541+0000","last_peered":"2024-06-02T15:40:00.032541+0000","last_clean":"2024-06-02T15:40:00.032541+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:00.032541+0000","last_undegraded":"2024-06-02T15:40:00.032541+0000","last_fullsized":"2024-06-02T15:40:00.032541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028256+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:40:00.028256+0000","last_peered":"2024-06-02T15:40:00.028256+0000","last_clean":"2024-06-02T15:40:00.028256+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:40:00.028256+0000","last_undegraded":"2024-06-02T15:40:00.028256+0000","last_fullsized":"2024-06-02T15:40:00.028256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031254+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031254+0000","last_peered":"2024-06-02T15:40:00.031254+0000","last_clean":"2024-06-02T15:40:00.031254+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031254+0000","last_undegraded":"2024-06-02T15:40:00.031254+0000","last_fullsized":"2024-06-02T15:40:00.031254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032414+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:00.032414+0000","last_peered":"2024-06-02T15:40:00.032414+0000","last_clean":"2024-06-02T15:40:00.032414+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:00.032414+0000","last_undegraded":"2024-06-02T15:40:00.032414+0000","last_fullsized":"2024-06-02T15:40:00.032414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024130+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:40:00.024130+0000","last_peered":"2024-06-02T15:40:00.024130+0000","last_clean":"2024-06-02T15:40:00.024130+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:40:00.024130+0000","last_undegraded":"2024-06-02T15:40:00.024130+0000","last_fullsized":"2024-06-02T15:40:00.024130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031205+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031205+0000","last_peered":"2024-06-02T15:40:00.031205+0000","last_clean":"2024-06-02T15:40:00.031205+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031205+0000","last_undegraded":"2024-06-02T15:40:00.031205+0000","last_fullsized":"2024-06-02T15:40:00.031205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:07.199573+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:40:07.199573+0000","last_peered":"2024-06-02T15:40:07.199573+0000","last_clean":"2024-06-02T15:40:07.199573+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:40:07.199573+0000","last_undegraded":"2024-06-02T15:40:07.199573+0000","last_fullsized":"2024-06-02T15:40:07.199573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032114+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:40:00.032114+0000","last_peered":"2024-06-02T15:40:00.032114+0000","last_clean":"2024-06-02T15:40:00.032114+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:40:00.032114+0000","last_undegraded":"2024-06-02T15:40:00.032114+0000","last_fullsized":"2024-06-02T15:40:00.032114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031104+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031104+0000","last_peered":"2024-06-02T15:40:00.031104+0000","last_clean":"2024-06-02T15:40:00.031104+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031104+0000","last_undegraded":"2024-06-02T15:40:00.031104+0000","last_fullsized":"2024-06-02T15:40:00.031104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031567+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:00.031567+0000","last_peered":"2024-06-02T15:40:00.031567+0000","last_clean":"2024-06-02T15:40:00.031567+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:00.031567+0000","last_undegraded":"2024-06-02T15:40:00.031567+0000","last_fullsized":"2024-06-02T15:40:00.031567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024400+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:40:00.024400+0000","last_peered":"2024-06-02T15:40:00.024400+0000","last_clean":"2024-06-02T15:40:00.024400+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:40:00.024400+0000","last_undegraded":"2024-06-02T15:40:00.024400+0000","last_fullsized":"2024-06-02T15:40:00.024400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031086+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:00.031086+0000","last_peered":"2024-06-02T15:40:00.031086+0000","last_clean":"2024-06-02T15:40:00.031086+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:00.031086+0000","last_undegraded":"2024-06-02T15:40:00.031086+0000","last_fullsized":"2024-06-02T15:40:00.031086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033944+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:00.033944+0000","last_peered":"2024-06-02T15:40:00.033944+0000","last_clean":"2024-06-02T15:40:00.033944+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:00.033944+0000","last_undegraded":"2024-06-02T15:40:00.033944+0000","last_fullsized":"2024-06-02T15:40:00.033944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032098+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:40:00.032098+0000","last_peered":"2024-06-02T15:40:00.032098+0000","last_clean":"2024-06-02T15:40:00.032098+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:40:00.032098+0000","last_undegraded":"2024-06-02T15:40:00.032098+0000","last_fullsized":"2024-06-02T15:40:00.032098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026054+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:00.026054+0000","last_peered":"2024-06-02T15:40:00.026054+0000","last_clean":"2024-06-02T15:40:00.026054+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:00.026054+0000","last_undegraded":"2024-06-02T15:40:00.026054+0000","last_fullsized":"2024-06-02T15:40:00.026054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023750+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:00.023750+0000","last_peered":"2024-06-02T15:40:00.023750+0000","last_clean":"2024-06-02T15:40:00.023750+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:00.023750+0000","last_undegraded":"2024-06-02T15:40:00.023750+0000","last_fullsized":"2024-06-02T15:40:00.023750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033582+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:40:00.033582+0000","last_peered":"2024-06-02T15:40:00.033582+0000","last_clean":"2024-06-02T15:40:00.033582+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:40:00.033582+0000","last_undegraded":"2024-06-02T15:40:00.033582+0000","last_fullsized":"2024-06-02T15:40:00.033582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031984+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:00.031984+0000","last_peered":"2024-06-02T15:40:00.031984+0000","last_clean":"2024-06-02T15:40:00.031984+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:00.031984+0000","last_undegraded":"2024-06-02T15:40:00.031984+0000","last_fullsized":"2024-06-02T15:40:00.031984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:06.197449+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:40:06.197449+0000","last_peered":"2024-06-02T15:40:06.197449+0000","last_clean":"2024-06-02T15:40:06.197449+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:40:06.197449+0000","last_undegraded":"2024-06-02T15:40:06.197449+0000","last_fullsized":"2024-06-02T15:40:06.197449+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029109+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:40:00.029109+0000","last_peered":"2024-06-02T15:40:00.029109+0000","last_clean":"2024-06-02T15:40:00.029109+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:40:00.029109+0000","last_undegraded":"2024-06-02T15:40:00.029109+0000","last_fullsized":"2024-06-02T15:40:00.029109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032870+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:00.032870+0000","last_peered":"2024-06-02T15:40:00.032870+0000","last_clean":"2024-06-02T15:40:00.032870+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:00.032870+0000","last_undegraded":"2024-06-02T15:40:00.032870+0000","last_fullsized":"2024-06-02T15:40:00.032870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027413+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:00.027413+0000","last_peered":"2024-06-02T15:40:00.027413+0000","last_clean":"2024-06-02T15:40:00.027413+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:00.027413+0000","last_undegraded":"2024-06-02T15:40:00.027413+0000","last_fullsized":"2024-06-02T15:40:00.027413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028862+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:40:00.028862+0000","last_peered":"2024-06-02T15:40:00.028862+0000","last_clean":"2024-06-02T15:40:00.028862+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:40:00.028862+0000","last_undegraded":"2024-06-02T15:40:00.028862+0000","last_fullsized":"2024-06-02T15:40:00.028862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031897+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:00.031897+0000","last_peered":"2024-06-02T15:40:00.031897+0000","last_clean":"2024-06-02T15:40:00.031897+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:00.031897+0000","last_undegraded":"2024-06-02T15:40:00.031897+0000","last_fullsized":"2024-06-02T15:40:00.031897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024069+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:00.024069+0000","last_peered":"2024-06-02T15:40:00.024069+0000","last_clean":"2024-06-02T15:40:00.024069+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:00.024069+0000","last_undegraded":"2024-06-02T15:40:00.024069+0000","last_fullsized":"2024-06-02T15:40:00.024069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028302+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:40:00.028302+0000","last_peered":"2024-06-02T15:40:00.028302+0000","last_clean":"2024-06-02T15:40:00.028302+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:40:00.028302+0000","last_undegraded":"2024-06-02T15:40:00.028302+0000","last_fullsized":"2024-06-02T15:40:00.028302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034356+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:00.034356+0000","last_peered":"2024-06-02T15:40:00.034356+0000","last_clean":"2024-06-02T15:40:00.034356+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:00.034356+0000","last_undegraded":"2024-06-02T15:40:00.034356+0000","last_fullsized":"2024-06-02T15:40:00.034356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:07.199235+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:40:07.199235+0000","last_peered":"2024-06-02T15:40:07.199235+0000","last_clean":"2024-06-02T15:40:07.199235+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:40:07.199235+0000","last_undegraded":"2024-06-02T15:40:07.199235+0000","last_fullsized":"2024-06-02T15:40:07.199235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64060,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":73,"num_read_kb":99,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032653+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:40:00.032653+0000","last_peered":"2024-06-02T15:40:00.032653+0000","last_clean":"2024-06-02T15:40:00.032653+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:40:00.032653+0000","last_undegraded":"2024-06-02T15:40:00.032653+0000","last_fullsized":"2024-06-02T15:40:00.032653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024621+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:00.024621+0000","last_peered":"2024-06-02T15:40:00.024621+0000","last_clean":"2024-06-02T15:40:00.024621+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:00.024621+0000","last_undegraded":"2024-06-02T15:40:00.024621+0000","last_fullsized":"2024-06-02T15:40:00.024621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030796+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:00.030796+0000","last_peered":"2024-06-02T15:40:00.030796+0000","last_clean":"2024-06-02T15:40:00.030796+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:00.030796+0000","last_undegraded":"2024-06-02T15:40:00.030796+0000","last_fullsized":"2024-06-02T15:40:00.030796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034303+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:40:00.034303+0000","last_peered":"2024-06-02T15:40:00.034303+0000","last_clean":"2024-06-02T15:40:00.034303+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:40:00.034303+0000","last_undegraded":"2024-06-02T15:40:00.034303+0000","last_fullsized":"2024-06-02T15:40:00.034303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029968+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:00.029968+0000","last_peered":"2024-06-02T15:40:00.029968+0000","last_clean":"2024-06-02T15:40:00.029968+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:00.029968+0000","last_undegraded":"2024-06-02T15:40:00.029968+0000","last_fullsized":"2024-06-02T15:40:00.029968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024002+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:00.024002+0000","last_peered":"2024-06-02T15:40:00.024002+0000","last_clean":"2024-06-02T15:40:00.024002+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:00.024002+0000","last_undegraded":"2024-06-02T15:40:00.024002+0000","last_fullsized":"2024-06-02T15:40:00.024002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027169+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:40:00.027169+0000","last_peered":"2024-06-02T15:40:00.027169+0000","last_clean":"2024-06-02T15:40:00.027169+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:40:00.027169+0000","last_undegraded":"2024-06-02T15:40:00.027169+0000","last_fullsized":"2024-06-02T15:40:00.027169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036451+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:00.036451+0000","last_peered":"2024-06-02T15:40:00.036451+0000","last_clean":"2024-06-02T15:40:00.036451+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:00.036451+0000","last_undegraded":"2024-06-02T15:40:00.036451+0000","last_fullsized":"2024-06-02T15:40:00.036451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:07.210781+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:40:07.210781+0000","last_peered":"2024-06-02T15:40:07.210781+0000","last_clean":"2024-06-02T15:40:07.210781+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:40:07.210781+0000","last_undegraded":"2024-06-02T15:40:07.210781+0000","last_fullsized":"2024-06-02T15:40:07.210781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026747+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:40:00.026747+0000","last_peered":"2024-06-02T15:40:00.026747+0000","last_clean":"2024-06-02T15:40:00.026747+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:40:00.026747+0000","last_undegraded":"2024-06-02T15:40:00.026747+0000","last_fullsized":"2024-06-02T15:40:00.026747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038308+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:00.038308+0000","last_peered":"2024-06-02T15:40:00.038308+0000","last_clean":"2024-06-02T15:40:00.038308+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:00.038308+0000","last_undegraded":"2024-06-02T15:40:00.038308+0000","last_fullsized":"2024-06-02T15:40:00.038308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031944+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:00.031944+0000","last_peered":"2024-06-02T15:40:00.031944+0000","last_clean":"2024-06-02T15:40:00.031944+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:00.031944+0000","last_undegraded":"2024-06-02T15:40:00.031944+0000","last_fullsized":"2024-06-02T15:40:00.031944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024153+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:40:00.024153+0000","last_peered":"2024-06-02T15:40:00.024153+0000","last_clean":"2024-06-02T15:40:00.024153+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:40:00.024153+0000","last_undegraded":"2024-06-02T15:40:00.024153+0000","last_fullsized":"2024-06-02T15:40:00.024153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023977+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:00.023977+0000","last_peered":"2024-06-02T15:40:00.023977+0000","last_clean":"2024-06-02T15:40:00.023977+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:00.023977+0000","last_undegraded":"2024-06-02T15:40:00.023977+0000","last_fullsized":"2024-06-02T15:40:00.023977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032758+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:00.032758+0000","last_peered":"2024-06-02T15:40:00.032758+0000","last_clean":"2024-06-02T15:40:00.032758+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:00.032758+0000","last_undegraded":"2024-06-02T15:40:00.032758+0000","last_fullsized":"2024-06-02T15:40:00.032758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030310+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:40:00.030310+0000","last_peered":"2024-06-02T15:40:00.030310+0000","last_clean":"2024-06-02T15:40:00.030310+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:40:00.030310+0000","last_undegraded":"2024-06-02T15:40:00.030310+0000","last_fullsized":"2024-06-02T15:40:00.030310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032026+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:00.032026+0000","last_peered":"2024-06-02T15:40:00.032026+0000","last_clean":"2024-06-02T15:40:00.032026+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:00.032026+0000","last_undegraded":"2024-06-02T15:40:00.032026+0000","last_fullsized":"2024-06-02T15:40:00.032026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035865+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:00.035865+0000","last_peered":"2024-06-02T15:40:00.035865+0000","last_clean":"2024-06-02T15:40:00.035865+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:00.035865+0000","last_undegraded":"2024-06-02T15:40:00.035865+0000","last_fullsized":"2024-06-02T15:40:00.035865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032339+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:40:00.032339+0000","last_peered":"2024-06-02T15:40:00.032339+0000","last_clean":"2024-06-02T15:40:00.032339+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:40:00.032339+0000","last_undegraded":"2024-06-02T15:40:00.032339+0000","last_fullsized":"2024-06-02T15:40:00.032339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034530+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:00.034530+0000","last_peered":"2024-06-02T15:40:00.034530+0000","last_clean":"2024-06-02T15:40:00.034530+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:00.034530+0000","last_undegraded":"2024-06-02T15:40:00.034530+0000","last_fullsized":"2024-06-02T15:40:00.034530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026869+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:00.026869+0000","last_peered":"2024-06-02T15:40:00.026869+0000","last_clean":"2024-06-02T15:40:00.026869+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:00.026869+0000","last_undegraded":"2024-06-02T15:40:00.026869+0000","last_fullsized":"2024-06-02T15:40:00.026869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031892+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:40:00.031892+0000","last_peered":"2024-06-02T15:40:00.031892+0000","last_clean":"2024-06-02T15:40:00.031892+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:40:00.031892+0000","last_undegraded":"2024-06-02T15:40:00.031892+0000","last_fullsized":"2024-06-02T15:40:00.031892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.041796+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:00.041796+0000","last_peered":"2024-06-02T15:40:00.041796+0000","last_clean":"2024-06-02T15:40:00.041796+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:00.041796+0000","last_undegraded":"2024-06-02T15:40:00.041796+0000","last_fullsized":"2024-06-02T15:40:00.041796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023944+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:40:00.023944+0000","last_peered":"2024-06-02T15:40:00.023944+0000","last_clean":"2024-06-02T15:40:00.023944+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:40:00.023944+0000","last_undegraded":"2024-06-02T15:40:00.023944+0000","last_fullsized":"2024-06-02T15:40:00.023944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:08.212020+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:40:08.212020+0000","last_peered":"2024-06-02T15:40:08.212020+0000","last_clean":"2024-06-02T15:40:08.212020+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:40:08.212020+0000","last_undegraded":"2024-06-02T15:40:08.212020+0000","last_fullsized":"2024-06-02T15:40:08.212020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.045344+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:00.045344+0000","last_peered":"2024-06-02T15:40:00.045344+0000","last_clean":"2024-06-02T15:40:00.045344+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:00.045344+0000","last_undegraded":"2024-06-02T15:40:00.045344+0000","last_fullsized":"2024-06-02T15:40:00.045344+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032522+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:40:00.032522+0000","last_peered":"2024-06-02T15:40:00.032522+0000","last_clean":"2024-06-02T15:40:00.032522+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:40:00.032522+0000","last_undegraded":"2024-06-02T15:40:00.032522+0000","last_fullsized":"2024-06-02T15:40:00.032522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031769+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:00.031769+0000","last_peered":"2024-06-02T15:40:00.031769+0000","last_clean":"2024-06-02T15:40:00.031769+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:00.031769+0000","last_undegraded":"2024-06-02T15:40:00.031769+0000","last_fullsized":"2024-06-02T15:40:00.031769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026545+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:40:00.026545+0000","last_peered":"2024-06-02T15:40:00.026545+0000","last_clean":"2024-06-02T15:40:00.026545+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:40:00.026545+0000","last_undegraded":"2024-06-02T15:40:00.026545+0000","last_fullsized":"2024-06-02T15:40:00.026545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024411+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:00.024411+0000","last_peered":"2024-06-02T15:40:00.024411+0000","last_clean":"2024-06-02T15:40:00.024411+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:00.024411+0000","last_undegraded":"2024-06-02T15:40:00.024411+0000","last_fullsized":"2024-06-02T15:40:00.024411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033728+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:40:00.033728+0000","last_peered":"2024-06-02T15:40:00.033728+0000","last_clean":"2024-06-02T15:40:00.033728+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:40:00.033728+0000","last_undegraded":"2024-06-02T15:40:00.033728+0000","last_fullsized":"2024-06-02T15:40:00.033728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026700+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:00.026700+0000","last_peered":"2024-06-02T15:40:00.026700+0000","last_clean":"2024-06-02T15:40:00.026700+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:00.026700+0000","last_undegraded":"2024-06-02T15:40:00.026700+0000","last_fullsized":"2024-06-02T15:40:00.026700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029847+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:00.029847+0000","last_peered":"2024-06-02T15:40:00.029847+0000","last_clean":"2024-06-02T15:40:00.029847+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:00.029847+0000","last_undegraded":"2024-06-02T15:40:00.029847+0000","last_fullsized":"2024-06-02T15:40:00.029847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032644+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:00.032644+0000","last_peered":"2024-06-02T15:40:00.032644+0000","last_clean":"2024-06-02T15:40:00.032644+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:00.032644+0000","last_undegraded":"2024-06-02T15:40:00.032644+0000","last_fullsized":"2024-06-02T15:40:00.032644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035006+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:40:00.035006+0000","last_peered":"2024-06-02T15:40:00.035006+0000","last_clean":"2024-06-02T15:40:00.035006+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:40:00.035006+0000","last_undegraded":"2024-06-02T15:40:00.035006+0000","last_fullsized":"2024-06-02T15:40:00.035006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031992+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:00.031992+0000","last_peered":"2024-06-02T15:40:00.031992+0000","last_clean":"2024-06-02T15:40:00.031992+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:00.031992+0000","last_undegraded":"2024-06-02T15:40:00.031992+0000","last_fullsized":"2024-06-02T15:40:00.031992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036345+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:40:00.036345+0000","last_peered":"2024-06-02T15:40:00.036345+0000","last_clean":"2024-06-02T15:40:00.036345+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:40:00.036345+0000","last_undegraded":"2024-06-02T15:40:00.036345+0000","last_fullsized":"2024-06-02T15:40:00.036345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032848+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:40:00.032848+0000","last_peered":"2024-06-02T15:40:00.032848+0000","last_clean":"2024-06-02T15:40:00.032848+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:40:00.032848+0000","last_undegraded":"2024-06-02T15:40:00.032848+0000","last_fullsized":"2024-06-02T15:40:00.032848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030517+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:00.030517+0000","last_peered":"2024-06-02T15:40:00.030517+0000","last_clean":"2024-06-02T15:40:00.030517+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:00.030517+0000","last_undegraded":"2024-06-02T15:40:00.030517+0000","last_fullsized":"2024-06-02T15:40:00.030517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031614+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:00.031614+0000","last_peered":"2024-06-02T15:40:00.031614+0000","last_clean":"2024-06-02T15:40:00.031614+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:00.031614+0000","last_undegraded":"2024-06-02T15:40:00.031614+0000","last_fullsized":"2024-06-02T15:40:00.031614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028585+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:40:00.028585+0000","last_peered":"2024-06-02T15:40:00.028585+0000","last_clean":"2024-06-02T15:40:00.028585+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:40:00.028585+0000","last_undegraded":"2024-06-02T15:40:00.028585+0000","last_fullsized":"2024-06-02T15:40:00.028585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031473+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:00.031473+0000","last_peered":"2024-06-02T15:40:00.031473+0000","last_clean":"2024-06-02T15:40:00.031473+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:00.031473+0000","last_undegraded":"2024-06-02T15:40:00.031473+0000","last_fullsized":"2024-06-02T15:40:00.031473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027595+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:00.027595+0000","last_peered":"2024-06-02T15:40:00.027595+0000","last_clean":"2024-06-02T15:40:00.027595+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:00.027595+0000","last_undegraded":"2024-06-02T15:40:00.027595+0000","last_fullsized":"2024-06-02T15:40:00.027595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031962+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:40:00.031962+0000","last_peered":"2024-06-02T15:40:00.031962+0000","last_clean":"2024-06-02T15:40:00.031962+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:40:00.031962+0000","last_undegraded":"2024-06-02T15:40:00.031962+0000","last_fullsized":"2024-06-02T15:40:00.031962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036615+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:00.036615+0000","last_peered":"2024-06-02T15:40:00.036615+0000","last_clean":"2024-06-02T15:40:00.036615+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:00.036615+0000","last_undegraded":"2024-06-02T15:40:00.036615+0000","last_fullsized":"2024-06-02T15:40:00.036615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032099+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:00.032099+0000","last_peered":"2024-06-02T15:40:00.032099+0000","last_clean":"2024-06-02T15:40:00.032099+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:00.032099+0000","last_undegraded":"2024-06-02T15:40:00.032099+0000","last_fullsized":"2024-06-02T15:40:00.032099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029548+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:40:00.029548+0000","last_peered":"2024-06-02T15:40:00.029548+0000","last_clean":"2024-06-02T15:40:00.029548+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:40:00.029548+0000","last_undegraded":"2024-06-02T15:40:00.029548+0000","last_fullsized":"2024-06-02T15:40:00.029548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032297+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:00.032297+0000","last_peered":"2024-06-02T15:40:00.032297+0000","last_clean":"2024-06-02T15:40:00.032297+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:00.032297+0000","last_undegraded":"2024-06-02T15:40:00.032297+0000","last_fullsized":"2024-06-02T15:40:00.032297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029712+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:40:00.029712+0000","last_peered":"2024-06-02T15:40:00.029712+0000","last_clean":"2024-06-02T15:40:00.029712+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:40:00.029712+0000","last_undegraded":"2024-06-02T15:40:00.029712+0000","last_fullsized":"2024-06-02T15:40:00.029712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033286+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:40:00.033286+0000","last_peered":"2024-06-02T15:40:00.033286+0000","last_clean":"2024-06-02T15:40:00.033286+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:40:00.033286+0000","last_undegraded":"2024-06-02T15:40:00.033286+0000","last_fullsized":"2024-06-02T15:40:00.033286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031434+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:00.031434+0000","last_peered":"2024-06-02T15:40:00.031434+0000","last_clean":"2024-06-02T15:40:00.031434+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:00.031434+0000","last_undegraded":"2024-06-02T15:40:00.031434+0000","last_fullsized":"2024-06-02T15:40:00.031434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032138+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:00.032138+0000","last_peered":"2024-06-02T15:40:00.032138+0000","last_clean":"2024-06-02T15:40:00.032138+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:00.032138+0000","last_undegraded":"2024-06-02T15:40:00.032138+0000","last_fullsized":"2024-06-02T15:40:00.032138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027001+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:40:00.027001+0000","last_peered":"2024-06-02T15:40:00.027001+0000","last_clean":"2024-06-02T15:40:00.027001+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:40:00.027001+0000","last_undegraded":"2024-06-02T15:40:00.027001+0000","last_fullsized":"2024-06-02T15:40:00.027001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027767+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:00.027767+0000","last_peered":"2024-06-02T15:40:00.027767+0000","last_clean":"2024-06-02T15:40:00.027767+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:00.027767+0000","last_undegraded":"2024-06-02T15:40:00.027767+0000","last_fullsized":"2024-06-02T15:40:00.027767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031020+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:00.031020+0000","last_peered":"2024-06-02T15:40:00.031020+0000","last_clean":"2024-06-02T15:40:00.031020+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:00.031020+0000","last_undegraded":"2024-06-02T15:40:00.031020+0000","last_fullsized":"2024-06-02T15:40:00.031020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027988+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:40:00.027988+0000","last_peered":"2024-06-02T15:40:00.027988+0000","last_clean":"2024-06-02T15:40:00.027988+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:40:00.027988+0000","last_undegraded":"2024-06-02T15:40:00.027988+0000","last_fullsized":"2024-06-02T15:40:00.027988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034227+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:00.034227+0000","last_peered":"2024-06-02T15:40:00.034227+0000","last_clean":"2024-06-02T15:40:00.034227+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:00.034227+0000","last_undegraded":"2024-06-02T15:40:00.034227+0000","last_fullsized":"2024-06-02T15:40:00.034227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024284+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:00.024284+0000","last_peered":"2024-06-02T15:40:00.024284+0000","last_clean":"2024-06-02T15:40:00.024284+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:00.024284+0000","last_undegraded":"2024-06-02T15:40:00.024284+0000","last_fullsized":"2024-06-02T15:40:00.024284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027395+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:40:00.027395+0000","last_peered":"2024-06-02T15:40:00.027395+0000","last_clean":"2024-06-02T15:40:00.027395+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:40:00.027395+0000","last_undegraded":"2024-06-02T15:40:00.027395+0000","last_fullsized":"2024-06-02T15:40:00.027395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023654+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:00.023654+0000","last_peered":"2024-06-02T15:40:00.023654+0000","last_clean":"2024-06-02T15:40:00.023654+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:00.023654+0000","last_undegraded":"2024-06-02T15:40:00.023654+0000","last_fullsized":"2024-06-02T15:40:00.023654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034978+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:00.034978+0000","last_peered":"2024-06-02T15:40:00.034978+0000","last_clean":"2024-06-02T15:40:00.034978+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:00.034978+0000","last_undegraded":"2024-06-02T15:40:00.034978+0000","last_fullsized":"2024-06-02T15:40:00.034978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027634+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:40:00.027634+0000","last_peered":"2024-06-02T15:40:00.027634+0000","last_clean":"2024-06-02T15:40:00.027634+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:40:00.027634+0000","last_undegraded":"2024-06-02T15:40:00.027634+0000","last_fullsized":"2024-06-02T15:40:00.027634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029555+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:40:00.029555+0000","last_peered":"2024-06-02T15:40:00.029555+0000","last_clean":"2024-06-02T15:40:00.029555+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:40:00.029555+0000","last_undegraded":"2024-06-02T15:40:00.029555+0000","last_fullsized":"2024-06-02T15:40:00.029555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030706+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:40:00.030706+0000","last_peered":"2024-06-02T15:40:00.030706+0000","last_clean":"2024-06-02T15:40:00.030706+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:40:00.030706+0000","last_undegraded":"2024-06-02T15:40:00.030706+0000","last_fullsized":"2024-06-02T15:40:00.030706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030392+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:40:00.030392+0000","last_peered":"2024-06-02T15:40:00.030392+0000","last_clean":"2024-06-02T15:40:00.030392+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:40:00.030392+0000","last_undegraded":"2024-06-02T15:40:00.030392+0000","last_fullsized":"2024-06-02T15:40:00.030392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026736+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:40:00.026736+0000","last_peered":"2024-06-02T15:40:00.026736+0000","last_clean":"2024-06-02T15:40:00.026736+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:40:00.026736+0000","last_undegraded":"2024-06-02T15:40:00.026736+0000","last_fullsized":"2024-06-02T15:40:00.026736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038178+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:40:00.038178+0000","last_peered":"2024-06-02T15:40:00.038178+0000","last_clean":"2024-06-02T15:40:00.038178+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:40:00.038178+0000","last_undegraded":"2024-06-02T15:40:00.038178+0000","last_fullsized":"2024-06-02T15:40:00.038178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030866+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:40:00.030866+0000","last_peered":"2024-06-02T15:40:00.030866+0000","last_clean":"2024-06-02T15:40:00.030866+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:40:00.030866+0000","last_undegraded":"2024-06-02T15:40:00.030866+0000","last_fullsized":"2024-06-02T15:40:00.030866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031038+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:40:00.031038+0000","last_peered":"2024-06-02T15:40:00.031038+0000","last_clean":"2024-06-02T15:40:00.031038+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:40:00.031038+0000","last_undegraded":"2024-06-02T15:40:00.031038+0000","last_fullsized":"2024-06-02T15:40:00.031038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033098+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:40:00.033098+0000","last_peered":"2024-06-02T15:40:00.033098+0000","last_clean":"2024-06-02T15:40:00.033098+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:40:00.033098+0000","last_undegraded":"2024-06-02T15:40:00.033098+0000","last_fullsized":"2024-06-02T15:40:00.033098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023785+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:40:00.023785+0000","last_peered":"2024-06-02T15:40:00.023785+0000","last_clean":"2024-06-02T15:40:00.023785+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:40:00.023785+0000","last_undegraded":"2024-06-02T15:40:00.023785+0000","last_fullsized":"2024-06-02T15:40:00.023785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032192+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:40:00.032192+0000","last_peered":"2024-06-02T15:40:00.032192+0000","last_clean":"2024-06-02T15:40:00.032192+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:40:00.032192+0000","last_undegraded":"2024-06-02T15:40:00.032192+0000","last_fullsized":"2024-06-02T15:40:00.032192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027945+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:40:00.027945+0000","last_peered":"2024-06-02T15:40:00.027945+0000","last_clean":"2024-06-02T15:40:00.027945+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:40:00.027945+0000","last_undegraded":"2024-06-02T15:40:00.027945+0000","last_fullsized":"2024-06-02T15:40:00.027945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034023+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:40:00.034023+0000","last_peered":"2024-06-02T15:40:00.034023+0000","last_clean":"2024-06-02T15:40:00.034023+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:40:00.034023+0000","last_undegraded":"2024-06-02T15:40:00.034023+0000","last_fullsized":"2024-06-02T15:40:00.034023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030141+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:40:00.030141+0000","last_peered":"2024-06-02T15:40:00.030141+0000","last_clean":"2024-06-02T15:40:00.030141+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:40:00.030141+0000","last_undegraded":"2024-06-02T15:40:00.030141+0000","last_fullsized":"2024-06-02T15:40:00.030141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":225,"num_read_kb":185,"num_write":169,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":201091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1720320,"data_stored":1165614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037810,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1114034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":573277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":515223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":582673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1646592,"data_stored":1105717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:40:12.305 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T15:40:12.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:12 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:40:12.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:12 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:40:12.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:12 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:40:12.746 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:13 smithi120 ceph-mon[36451]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:13 smithi120 ceph-mon[36451]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-02T15:40:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:13 smithi135 ceph-mon[36839]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:13 smithi135 ceph-mon[36839]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-02T15:40:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:13 smithi090 ceph-mon[31595]: from='client.24721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:13 smithi090 ceph-mon[31595]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-06-02T15:40:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:14 smithi090 ceph-mon[31595]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:40:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:14 smithi120 ceph-mon[36451]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:40:14.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:14.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:14 smithi135 ceph-mon[36839]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:40:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 -- 172.21.15.90:0/912727493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 msgr2=0x7f5ce4109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/912727493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce4109f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009990 tx=0x7f5ccc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 -- 172.21.15.90:0/912727493 shutdown_connections 2024-06-02T15:40:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/912727493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410a440 0x7f5ce41168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/912727493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410e950 0x7f5ce410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/912727493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce4109f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 -- 172.21.15.90:0/912727493 >> 172.21.15.90:0/912727493 conn(0x7f5ce40792d0 msgr2=0x7f5ce4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:14.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 -- 172.21.15.90:0/912727493 shutdown_connections 2024-06-02T15:40:14.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.994+0000 7f5ce8e00700 1 -- 172.21.15.90:0/912727493 wait complete. 2024-06-02T15:40:14.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.995+0000 7f5ce8e00700 1 Processor -- start 2024-06-02T15:40:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.995+0000 7f5ce8e00700 1 -- start start 2024-06-02T15:40:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410a440 0x7f5ce41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 0x7f5ce41a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4119c80 con 0x7f5ce4109a80 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4119b00 con 0x7f5ce410a440 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce8e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce4119e00 con 0x7f5ce410e950 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 0x7f5ce41a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57628/0 (socket says 172.21.15.90:57628) 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5ce3400700 1 -- 172.21.15.90:0/194240127 learned_addr learned my addr 172.21.15.90:0/194240127 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.996+0000 7f5cdbe00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410a440 0x7f5ce41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 -- 172.21.15.90:0/194240127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410a440 msgr2=0x7f5ce41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410a440 0x7f5ce41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 -- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 msgr2=0x7f5ce41a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 -- 172.21.15.90:0/194240127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce41a5160 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3400700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce3e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 0x7f5ce41a4a20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc007f10 tx=0x7f5cdc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc004db0 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cdc009e20 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce41a5450 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc015400 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.997+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce41accd0 con 0x7f5ce410e950 2024-06-02T15:40:14.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:14.998+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce406a230 con 0x7f5ce410e950 2024-06-02T15:40:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.001+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5cdc005260 con 0x7f5ce410e950 2024-06-02T15:40:15.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.003+0000 7f5ce0c00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 0x7f5cc80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:15.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.003+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5cdc0e2590 con 0x7f5ce410e950 2024-06-02T15:40:15.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.003+0000 7f5ce3400700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 0x7f5cc80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:15.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.004+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cdc010070 con 0x7f5ce410e950 2024-06-02T15:40:15.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.004+0000 7f5ce3400700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 0x7f5cc80b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011550 tx=0x7f5cd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:15.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.231+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5ce4063980 con 0x7f5cc80b2990 2024-06-02T15:40:15.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.243+0000 7f5ce0c00700 1 -- 172.21.15.90:0/194240127 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340366 (secure 0 0 0) 0x7f5ce4063980 con 0x7f5cc80b2990 2024-06-02T15:40:15.248 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 msgr2=0x7f5cc80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 0x7f5cc80b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4011550 tx=0x7f5cd4009260 comp rx=0 tx=0).stop 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 msgr2=0x7f5ce41a4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 0x7f5ce41a4a20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc007f10 tx=0x7f5cdc00d3b0 comp rx=0 tx=0).stop 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 shutdown_connections 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5cc80b2990 0x7f5cc80b4e50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ce410e950 0x7f5ce41a4a20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ce4109a80 0x7f5ce41a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:15.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 --2- 172.21.15.90:0/194240127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5ce410a440 0x7f5ce41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:15.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 >> 172.21.15.90:0/194240127 conn(0x7f5ce40792d0 msgr2=0x7f5ce40798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:15.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 shutdown_connections 2024-06-02T15:40:15.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:15.252+0000 7f5ce8e00700 1 -- 172.21.15.90:0/194240127 wait complete. 2024-06-02T15:40:15.252 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:40:15.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:15 smithi090 ceph-mon[31595]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:16.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:15 smithi120 ceph-mon[36451]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:16.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:15 smithi135 ceph-mon[36839]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:40:16.616 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-06-02T15:40:13.733653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659054,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":316,"num_read_kb":259,"num_write":296,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375868,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565956,"statfs":{"total":1151940427776,"available":1151555538944,"internally_reserved":0,"allocated":14532608,"data_stored":8171012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9083,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":50,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006120"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031464+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:40:00.031464+0000","last_peered":"2024-06-02T15:40:00.031464+0000","last_clean":"2024-06-02T15:40:00.031464+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:40:00.031464+0000","last_undegraded":"2024-06-02T15:40:00.031464+0000","last_fullsized":"2024-06-02T15:40:00.031464+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028321+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:40:00.028321+0000","last_peered":"2024-06-02T15:40:00.028321+0000","last_clean":"2024-06-02T15:40:00.028321+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:40:00.028321+0000","last_undegraded":"2024-06-02T15:40:00.028321+0000","last_fullsized":"2024-06-02T15:40:00.028321+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026888+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:40:00.026888+0000","last_peered":"2024-06-02T15:40:00.026888+0000","last_clean":"2024-06-02T15:40:00.026888+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:40:00.026888+0000","last_undegraded":"2024-06-02T15:40:00.026888+0000","last_fullsized":"2024-06-02T15:40:00.026888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031981+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:40:00.031981+0000","last_peered":"2024-06-02T15:40:00.031981+0000","last_clean":"2024-06-02T15:40:00.031981+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:40:00.031981+0000","last_undegraded":"2024-06-02T15:40:00.031981+0000","last_fullsized":"2024-06-02T15:40:00.031981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032260+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:40:00.032260+0000","last_peered":"2024-06-02T15:40:00.032260+0000","last_clean":"2024-06-02T15:40:00.032260+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:40:00.032260+0000","last_undegraded":"2024-06-02T15:40:00.032260+0000","last_fullsized":"2024-06-02T15:40:00.032260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031691+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:40:00.031691+0000","last_peered":"2024-06-02T15:40:00.031691+0000","last_clean":"2024-06-02T15:40:00.031691+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:40:00.031691+0000","last_undegraded":"2024-06-02T15:40:00.031691+0000","last_fullsized":"2024-06-02T15:40:00.031691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026768+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:40:00.026768+0000","last_peered":"2024-06-02T15:40:00.026768+0000","last_clean":"2024-06-02T15:40:00.026768+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:40:00.026768+0000","last_undegraded":"2024-06-02T15:40:00.026768+0000","last_fullsized":"2024-06-02T15:40:00.026768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032189+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:40:00.032189+0000","last_peered":"2024-06-02T15:40:00.032189+0000","last_clean":"2024-06-02T15:40:00.032189+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:40:00.032189+0000","last_undegraded":"2024-06-02T15:40:00.032189+0000","last_fullsized":"2024-06-02T15:40:00.032189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028782+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:40:00.028782+0000","last_peered":"2024-06-02T15:40:00.028782+0000","last_clean":"2024-06-02T15:40:00.028782+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:40:00.028782+0000","last_undegraded":"2024-06-02T15:40:00.028782+0000","last_fullsized":"2024-06-02T15:40:00.028782+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032137+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:40:00.032137+0000","last_peered":"2024-06-02T15:40:00.032137+0000","last_clean":"2024-06-02T15:40:00.032137+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:40:00.032137+0000","last_undegraded":"2024-06-02T15:40:00.032137+0000","last_fullsized":"2024-06-02T15:40:00.032137+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027808+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:40:00.027808+0000","last_peered":"2024-06-02T15:40:00.027808+0000","last_clean":"2024-06-02T15:40:00.027808+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:40:00.027808+0000","last_undegraded":"2024-06-02T15:40:00.027808+0000","last_fullsized":"2024-06-02T15:40:00.027808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031351+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:40:00.031351+0000","last_peered":"2024-06-02T15:40:00.031351+0000","last_clean":"2024-06-02T15:40:00.031351+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:40:00.031351+0000","last_undegraded":"2024-06-02T15:40:00.031351+0000","last_fullsized":"2024-06-02T15:40:00.031351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034138+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:40:00.034138+0000","last_peered":"2024-06-02T15:40:00.034138+0000","last_clean":"2024-06-02T15:40:00.034138+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:40:00.034138+0000","last_undegraded":"2024-06-02T15:40:00.034138+0000","last_fullsized":"2024-06-02T15:40:00.034138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026409+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:40:00.026409+0000","last_peered":"2024-06-02T15:40:00.026409+0000","last_clean":"2024-06-02T15:40:00.026409+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:40:00.026409+0000","last_undegraded":"2024-06-02T15:40:00.026409+0000","last_fullsized":"2024-06-02T15:40:00.026409+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026214+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:40:00.026214+0000","last_peered":"2024-06-02T15:40:00.026214+0000","last_clean":"2024-06-02T15:40:00.026214+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:40:00.026214+0000","last_undegraded":"2024-06-02T15:40:00.026214+0000","last_fullsized":"2024-06-02T15:40:00.026214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027558+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:40:00.027558+0000","last_peered":"2024-06-02T15:40:00.027558+0000","last_clean":"2024-06-02T15:40:00.027558+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:40:00.027558+0000","last_undegraded":"2024-06-02T15:40:00.027558+0000","last_fullsized":"2024-06-02T15:40:00.027558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031210+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:40:00.031210+0000","last_peered":"2024-06-02T15:40:00.031210+0000","last_clean":"2024-06-02T15:40:00.031210+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:40:00.031210+0000","last_undegraded":"2024-06-02T15:40:00.031210+0000","last_fullsized":"2024-06-02T15:40:00.031210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036187+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:40:00.036187+0000","last_peered":"2024-06-02T15:40:00.036187+0000","last_clean":"2024-06-02T15:40:00.036187+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:40:00.036187+0000","last_undegraded":"2024-06-02T15:40:00.036187+0000","last_fullsized":"2024-06-02T15:40:00.036187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031209+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:40:00.031209+0000","last_peered":"2024-06-02T15:40:00.031209+0000","last_clean":"2024-06-02T15:40:00.031209+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:40:00.031209+0000","last_undegraded":"2024-06-02T15:40:00.031209+0000","last_fullsized":"2024-06-02T15:40:00.031209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030505+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:00.030505+0000","last_peered":"2024-06-02T15:40:00.030505+0000","last_clean":"2024-06-02T15:40:00.030505+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:00.030505+0000","last_undegraded":"2024-06-02T15:40:00.030505+0000","last_fullsized":"2024-06-02T15:40:00.030505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036698+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:00.036698+0000","last_peered":"2024-06-02T15:40:00.036698+0000","last_clean":"2024-06-02T15:40:00.036698+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:00.036698+0000","last_undegraded":"2024-06-02T15:40:00.036698+0000","last_fullsized":"2024-06-02T15:40:00.036698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034575+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:40:00.034575+0000","last_peered":"2024-06-02T15:40:00.034575+0000","last_clean":"2024-06-02T15:40:00.034575+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:40:00.034575+0000","last_undegraded":"2024-06-02T15:40:00.034575+0000","last_fullsized":"2024-06-02T15:40:00.034575+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.041841+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:00.041841+0000","last_peered":"2024-06-02T15:40:00.041841+0000","last_clean":"2024-06-02T15:40:00.041841+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:00.041841+0000","last_undegraded":"2024-06-02T15:40:00.041841+0000","last_fullsized":"2024-06-02T15:40:00.041841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034821+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:00.034821+0000","last_peered":"2024-06-02T15:40:00.034821+0000","last_clean":"2024-06-02T15:40:00.034821+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:00.034821+0000","last_undegraded":"2024-06-02T15:40:00.034821+0000","last_fullsized":"2024-06-02T15:40:00.034821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029136+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:40:00.029136+0000","last_peered":"2024-06-02T15:40:00.029136+0000","last_clean":"2024-06-02T15:40:00.029136+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:40:00.029136+0000","last_undegraded":"2024-06-02T15:40:00.029136+0000","last_fullsized":"2024-06-02T15:40:00.029136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031380+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031380+0000","last_peered":"2024-06-02T15:40:00.031380+0000","last_clean":"2024-06-02T15:40:00.031380+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031380+0000","last_undegraded":"2024-06-02T15:40:00.031380+0000","last_fullsized":"2024-06-02T15:40:00.031380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032541+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:00.032541+0000","last_peered":"2024-06-02T15:40:00.032541+0000","last_clean":"2024-06-02T15:40:00.032541+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:00.032541+0000","last_undegraded":"2024-06-02T15:40:00.032541+0000","last_fullsized":"2024-06-02T15:40:00.032541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028256+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:40:00.028256+0000","last_peered":"2024-06-02T15:40:00.028256+0000","last_clean":"2024-06-02T15:40:00.028256+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:40:00.028256+0000","last_undegraded":"2024-06-02T15:40:00.028256+0000","last_fullsized":"2024-06-02T15:40:00.028256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031254+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031254+0000","last_peered":"2024-06-02T15:40:00.031254+0000","last_clean":"2024-06-02T15:40:00.031254+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031254+0000","last_undegraded":"2024-06-02T15:40:00.031254+0000","last_fullsized":"2024-06-02T15:40:00.031254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032414+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:00.032414+0000","last_peered":"2024-06-02T15:40:00.032414+0000","last_clean":"2024-06-02T15:40:00.032414+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:00.032414+0000","last_undegraded":"2024-06-02T15:40:00.032414+0000","last_fullsized":"2024-06-02T15:40:00.032414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024130+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:40:00.024130+0000","last_peered":"2024-06-02T15:40:00.024130+0000","last_clean":"2024-06-02T15:40:00.024130+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:40:00.024130+0000","last_undegraded":"2024-06-02T15:40:00.024130+0000","last_fullsized":"2024-06-02T15:40:00.024130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031205+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031205+0000","last_peered":"2024-06-02T15:40:00.031205+0000","last_clean":"2024-06-02T15:40:00.031205+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031205+0000","last_undegraded":"2024-06-02T15:40:00.031205+0000","last_fullsized":"2024-06-02T15:40:00.031205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:07.199573+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:40:07.199573+0000","last_peered":"2024-06-02T15:40:07.199573+0000","last_clean":"2024-06-02T15:40:07.199573+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:40:07.199573+0000","last_undegraded":"2024-06-02T15:40:07.199573+0000","last_fullsized":"2024-06-02T15:40:07.199573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032114+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:40:00.032114+0000","last_peered":"2024-06-02T15:40:00.032114+0000","last_clean":"2024-06-02T15:40:00.032114+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:40:00.032114+0000","last_undegraded":"2024-06-02T15:40:00.032114+0000","last_fullsized":"2024-06-02T15:40:00.032114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031104+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031104+0000","last_peered":"2024-06-02T15:40:00.031104+0000","last_clean":"2024-06-02T15:40:00.031104+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031104+0000","last_undegraded":"2024-06-02T15:40:00.031104+0000","last_fullsized":"2024-06-02T15:40:00.031104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031567+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:00.031567+0000","last_peered":"2024-06-02T15:40:00.031567+0000","last_clean":"2024-06-02T15:40:00.031567+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:00.031567+0000","last_undegraded":"2024-06-02T15:40:00.031567+0000","last_fullsized":"2024-06-02T15:40:00.031567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024400+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:40:00.024400+0000","last_peered":"2024-06-02T15:40:00.024400+0000","last_clean":"2024-06-02T15:40:00.024400+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:40:00.024400+0000","last_undegraded":"2024-06-02T15:40:00.024400+0000","last_fullsized":"2024-06-02T15:40:00.024400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031086+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:00.031086+0000","last_peered":"2024-06-02T15:40:00.031086+0000","last_clean":"2024-06-02T15:40:00.031086+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:00.031086+0000","last_undegraded":"2024-06-02T15:40:00.031086+0000","last_fullsized":"2024-06-02T15:40:00.031086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033944+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:00.033944+0000","last_peered":"2024-06-02T15:40:00.033944+0000","last_clean":"2024-06-02T15:40:00.033944+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:00.033944+0000","last_undegraded":"2024-06-02T15:40:00.033944+0000","last_fullsized":"2024-06-02T15:40:00.033944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032098+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:40:00.032098+0000","last_peered":"2024-06-02T15:40:00.032098+0000","last_clean":"2024-06-02T15:40:00.032098+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:40:00.032098+0000","last_undegraded":"2024-06-02T15:40:00.032098+0000","last_fullsized":"2024-06-02T15:40:00.032098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026054+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:00.026054+0000","last_peered":"2024-06-02T15:40:00.026054+0000","last_clean":"2024-06-02T15:40:00.026054+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:00.026054+0000","last_undegraded":"2024-06-02T15:40:00.026054+0000","last_fullsized":"2024-06-02T15:40:00.026054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023750+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:00.023750+0000","last_peered":"2024-06-02T15:40:00.023750+0000","last_clean":"2024-06-02T15:40:00.023750+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:00.023750+0000","last_undegraded":"2024-06-02T15:40:00.023750+0000","last_fullsized":"2024-06-02T15:40:00.023750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033582+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:40:00.033582+0000","last_peered":"2024-06-02T15:40:00.033582+0000","last_clean":"2024-06-02T15:40:00.033582+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:40:00.033582+0000","last_undegraded":"2024-06-02T15:40:00.033582+0000","last_fullsized":"2024-06-02T15:40:00.033582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031984+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:00.031984+0000","last_peered":"2024-06-02T15:40:00.031984+0000","last_clean":"2024-06-02T15:40:00.031984+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:00.031984+0000","last_undegraded":"2024-06-02T15:40:00.031984+0000","last_fullsized":"2024-06-02T15:40:00.031984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:11.205354+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:40:11.205354+0000","last_peered":"2024-06-02T15:40:11.205354+0000","last_clean":"2024-06-02T15:40:11.205354+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:40:11.205354+0000","last_undegraded":"2024-06-02T15:40:11.205354+0000","last_fullsized":"2024-06-02T15:40:11.205354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029109+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:40:00.029109+0000","last_peered":"2024-06-02T15:40:00.029109+0000","last_clean":"2024-06-02T15:40:00.029109+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:40:00.029109+0000","last_undegraded":"2024-06-02T15:40:00.029109+0000","last_fullsized":"2024-06-02T15:40:00.029109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032870+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:00.032870+0000","last_peered":"2024-06-02T15:40:00.032870+0000","last_clean":"2024-06-02T15:40:00.032870+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:00.032870+0000","last_undegraded":"2024-06-02T15:40:00.032870+0000","last_fullsized":"2024-06-02T15:40:00.032870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027413+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:00.027413+0000","last_peered":"2024-06-02T15:40:00.027413+0000","last_clean":"2024-06-02T15:40:00.027413+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:00.027413+0000","last_undegraded":"2024-06-02T15:40:00.027413+0000","last_fullsized":"2024-06-02T15:40:00.027413+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028862+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:40:00.028862+0000","last_peered":"2024-06-02T15:40:00.028862+0000","last_clean":"2024-06-02T15:40:00.028862+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:40:00.028862+0000","last_undegraded":"2024-06-02T15:40:00.028862+0000","last_fullsized":"2024-06-02T15:40:00.028862+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031897+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:00.031897+0000","last_peered":"2024-06-02T15:40:00.031897+0000","last_clean":"2024-06-02T15:40:00.031897+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:00.031897+0000","last_undegraded":"2024-06-02T15:40:00.031897+0000","last_fullsized":"2024-06-02T15:40:00.031897+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024069+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:00.024069+0000","last_peered":"2024-06-02T15:40:00.024069+0000","last_clean":"2024-06-02T15:40:00.024069+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:00.024069+0000","last_undegraded":"2024-06-02T15:40:00.024069+0000","last_fullsized":"2024-06-02T15:40:00.024069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028302+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:40:00.028302+0000","last_peered":"2024-06-02T15:40:00.028302+0000","last_clean":"2024-06-02T15:40:00.028302+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:40:00.028302+0000","last_undegraded":"2024-06-02T15:40:00.028302+0000","last_fullsized":"2024-06-02T15:40:00.028302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034356+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:00.034356+0000","last_peered":"2024-06-02T15:40:00.034356+0000","last_clean":"2024-06-02T15:40:00.034356+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:00.034356+0000","last_undegraded":"2024-06-02T15:40:00.034356+0000","last_fullsized":"2024-06-02T15:40:00.034356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'34","reported_seq":177,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:12.206487+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:40:12.206487+0000","last_peered":"2024-06-02T15:40:12.206487+0000","last_clean":"2024-06-02T15:40:12.206487+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:40:12.206487+0000","last_undegraded":"2024-06-02T15:40:12.206487+0000","last_fullsized":"2024-06-02T15:40:12.206487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64060,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":83,"num_read_kb":104,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032653+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:40:00.032653+0000","last_peered":"2024-06-02T15:40:00.032653+0000","last_clean":"2024-06-02T15:40:00.032653+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:40:00.032653+0000","last_undegraded":"2024-06-02T15:40:00.032653+0000","last_fullsized":"2024-06-02T15:40:00.032653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024621+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:00.024621+0000","last_peered":"2024-06-02T15:40:00.024621+0000","last_clean":"2024-06-02T15:40:00.024621+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:00.024621+0000","last_undegraded":"2024-06-02T15:40:00.024621+0000","last_fullsized":"2024-06-02T15:40:00.024621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030796+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:00.030796+0000","last_peered":"2024-06-02T15:40:00.030796+0000","last_clean":"2024-06-02T15:40:00.030796+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:00.030796+0000","last_undegraded":"2024-06-02T15:40:00.030796+0000","last_fullsized":"2024-06-02T15:40:00.030796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034303+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:40:00.034303+0000","last_peered":"2024-06-02T15:40:00.034303+0000","last_clean":"2024-06-02T15:40:00.034303+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:40:00.034303+0000","last_undegraded":"2024-06-02T15:40:00.034303+0000","last_fullsized":"2024-06-02T15:40:00.034303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029968+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:00.029968+0000","last_peered":"2024-06-02T15:40:00.029968+0000","last_clean":"2024-06-02T15:40:00.029968+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:00.029968+0000","last_undegraded":"2024-06-02T15:40:00.029968+0000","last_fullsized":"2024-06-02T15:40:00.029968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024002+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:00.024002+0000","last_peered":"2024-06-02T15:40:00.024002+0000","last_clean":"2024-06-02T15:40:00.024002+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:00.024002+0000","last_undegraded":"2024-06-02T15:40:00.024002+0000","last_fullsized":"2024-06-02T15:40:00.024002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027169+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:40:00.027169+0000","last_peered":"2024-06-02T15:40:00.027169+0000","last_clean":"2024-06-02T15:40:00.027169+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:40:00.027169+0000","last_undegraded":"2024-06-02T15:40:00.027169+0000","last_fullsized":"2024-06-02T15:40:00.027169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036451+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:00.036451+0000","last_peered":"2024-06-02T15:40:00.036451+0000","last_clean":"2024-06-02T15:40:00.036451+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:00.036451+0000","last_undegraded":"2024-06-02T15:40:00.036451+0000","last_fullsized":"2024-06-02T15:40:00.036451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:07.210781+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:40:07.210781+0000","last_peered":"2024-06-02T15:40:07.210781+0000","last_clean":"2024-06-02T15:40:07.210781+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:40:07.210781+0000","last_undegraded":"2024-06-02T15:40:07.210781+0000","last_fullsized":"2024-06-02T15:40:07.210781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026747+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:40:00.026747+0000","last_peered":"2024-06-02T15:40:00.026747+0000","last_clean":"2024-06-02T15:40:00.026747+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:40:00.026747+0000","last_undegraded":"2024-06-02T15:40:00.026747+0000","last_fullsized":"2024-06-02T15:40:00.026747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038308+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:00.038308+0000","last_peered":"2024-06-02T15:40:00.038308+0000","last_clean":"2024-06-02T15:40:00.038308+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:00.038308+0000","last_undegraded":"2024-06-02T15:40:00.038308+0000","last_fullsized":"2024-06-02T15:40:00.038308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031944+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:00.031944+0000","last_peered":"2024-06-02T15:40:00.031944+0000","last_clean":"2024-06-02T15:40:00.031944+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:00.031944+0000","last_undegraded":"2024-06-02T15:40:00.031944+0000","last_fullsized":"2024-06-02T15:40:00.031944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024153+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:40:00.024153+0000","last_peered":"2024-06-02T15:40:00.024153+0000","last_clean":"2024-06-02T15:40:00.024153+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:40:00.024153+0000","last_undegraded":"2024-06-02T15:40:00.024153+0000","last_fullsized":"2024-06-02T15:40:00.024153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023977+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:00.023977+0000","last_peered":"2024-06-02T15:40:00.023977+0000","last_clean":"2024-06-02T15:40:00.023977+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:00.023977+0000","last_undegraded":"2024-06-02T15:40:00.023977+0000","last_fullsized":"2024-06-02T15:40:00.023977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032758+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:00.032758+0000","last_peered":"2024-06-02T15:40:00.032758+0000","last_clean":"2024-06-02T15:40:00.032758+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:00.032758+0000","last_undegraded":"2024-06-02T15:40:00.032758+0000","last_fullsized":"2024-06-02T15:40:00.032758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030310+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:40:00.030310+0000","last_peered":"2024-06-02T15:40:00.030310+0000","last_clean":"2024-06-02T15:40:00.030310+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:40:00.030310+0000","last_undegraded":"2024-06-02T15:40:00.030310+0000","last_fullsized":"2024-06-02T15:40:00.030310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032026+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:00.032026+0000","last_peered":"2024-06-02T15:40:00.032026+0000","last_clean":"2024-06-02T15:40:00.032026+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:00.032026+0000","last_undegraded":"2024-06-02T15:40:00.032026+0000","last_fullsized":"2024-06-02T15:40:00.032026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035865+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:00.035865+0000","last_peered":"2024-06-02T15:40:00.035865+0000","last_clean":"2024-06-02T15:40:00.035865+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:00.035865+0000","last_undegraded":"2024-06-02T15:40:00.035865+0000","last_fullsized":"2024-06-02T15:40:00.035865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032339+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:40:00.032339+0000","last_peered":"2024-06-02T15:40:00.032339+0000","last_clean":"2024-06-02T15:40:00.032339+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:40:00.032339+0000","last_undegraded":"2024-06-02T15:40:00.032339+0000","last_fullsized":"2024-06-02T15:40:00.032339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034530+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:00.034530+0000","last_peered":"2024-06-02T15:40:00.034530+0000","last_clean":"2024-06-02T15:40:00.034530+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:00.034530+0000","last_undegraded":"2024-06-02T15:40:00.034530+0000","last_fullsized":"2024-06-02T15:40:00.034530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026869+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:00.026869+0000","last_peered":"2024-06-02T15:40:00.026869+0000","last_clean":"2024-06-02T15:40:00.026869+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:00.026869+0000","last_undegraded":"2024-06-02T15:40:00.026869+0000","last_fullsized":"2024-06-02T15:40:00.026869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031892+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:40:00.031892+0000","last_peered":"2024-06-02T15:40:00.031892+0000","last_clean":"2024-06-02T15:40:00.031892+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:40:00.031892+0000","last_undegraded":"2024-06-02T15:40:00.031892+0000","last_fullsized":"2024-06-02T15:40:00.031892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.041796+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:00.041796+0000","last_peered":"2024-06-02T15:40:00.041796+0000","last_clean":"2024-06-02T15:40:00.041796+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:00.041796+0000","last_undegraded":"2024-06-02T15:40:00.041796+0000","last_fullsized":"2024-06-02T15:40:00.041796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023944+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:40:00.023944+0000","last_peered":"2024-06-02T15:40:00.023944+0000","last_clean":"2024-06-02T15:40:00.023944+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:40:00.023944+0000","last_undegraded":"2024-06-02T15:40:00.023944+0000","last_fullsized":"2024-06-02T15:40:00.023944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:08.212020+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:40:08.212020+0000","last_peered":"2024-06-02T15:40:08.212020+0000","last_clean":"2024-06-02T15:40:08.212020+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:40:08.212020+0000","last_undegraded":"2024-06-02T15:40:08.212020+0000","last_fullsized":"2024-06-02T15:40:08.212020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.045344+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:00.045344+0000","last_peered":"2024-06-02T15:40:00.045344+0000","last_clean":"2024-06-02T15:40:00.045344+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:00.045344+0000","last_undegraded":"2024-06-02T15:40:00.045344+0000","last_fullsized":"2024-06-02T15:40:00.045344+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032522+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:40:00.032522+0000","last_peered":"2024-06-02T15:40:00.032522+0000","last_clean":"2024-06-02T15:40:00.032522+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:40:00.032522+0000","last_undegraded":"2024-06-02T15:40:00.032522+0000","last_fullsized":"2024-06-02T15:40:00.032522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031769+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:00.031769+0000","last_peered":"2024-06-02T15:40:00.031769+0000","last_clean":"2024-06-02T15:40:00.031769+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:00.031769+0000","last_undegraded":"2024-06-02T15:40:00.031769+0000","last_fullsized":"2024-06-02T15:40:00.031769+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026545+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:40:00.026545+0000","last_peered":"2024-06-02T15:40:00.026545+0000","last_clean":"2024-06-02T15:40:00.026545+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:40:00.026545+0000","last_undegraded":"2024-06-02T15:40:00.026545+0000","last_fullsized":"2024-06-02T15:40:00.026545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024411+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:00.024411+0000","last_peered":"2024-06-02T15:40:00.024411+0000","last_clean":"2024-06-02T15:40:00.024411+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:00.024411+0000","last_undegraded":"2024-06-02T15:40:00.024411+0000","last_fullsized":"2024-06-02T15:40:00.024411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033728+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:40:00.033728+0000","last_peered":"2024-06-02T15:40:00.033728+0000","last_clean":"2024-06-02T15:40:00.033728+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:40:00.033728+0000","last_undegraded":"2024-06-02T15:40:00.033728+0000","last_fullsized":"2024-06-02T15:40:00.033728+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026700+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:00.026700+0000","last_peered":"2024-06-02T15:40:00.026700+0000","last_clean":"2024-06-02T15:40:00.026700+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:00.026700+0000","last_undegraded":"2024-06-02T15:40:00.026700+0000","last_fullsized":"2024-06-02T15:40:00.026700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029847+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:00.029847+0000","last_peered":"2024-06-02T15:40:00.029847+0000","last_clean":"2024-06-02T15:40:00.029847+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:00.029847+0000","last_undegraded":"2024-06-02T15:40:00.029847+0000","last_fullsized":"2024-06-02T15:40:00.029847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032644+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:00.032644+0000","last_peered":"2024-06-02T15:40:00.032644+0000","last_clean":"2024-06-02T15:40:00.032644+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:00.032644+0000","last_undegraded":"2024-06-02T15:40:00.032644+0000","last_fullsized":"2024-06-02T15:40:00.032644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035006+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:40:00.035006+0000","last_peered":"2024-06-02T15:40:00.035006+0000","last_clean":"2024-06-02T15:40:00.035006+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:40:00.035006+0000","last_undegraded":"2024-06-02T15:40:00.035006+0000","last_fullsized":"2024-06-02T15:40:00.035006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031992+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:00.031992+0000","last_peered":"2024-06-02T15:40:00.031992+0000","last_clean":"2024-06-02T15:40:00.031992+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:00.031992+0000","last_undegraded":"2024-06-02T15:40:00.031992+0000","last_fullsized":"2024-06-02T15:40:00.031992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:02.501959+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:40:02.501959+0000","last_peered":"2024-06-02T15:40:02.501959+0000","last_clean":"2024-06-02T15:40:02.501959+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:40:02.501959+0000","last_undegraded":"2024-06-02T15:40:02.501959+0000","last_fullsized":"2024-06-02T15:40:02.501959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032848+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:40:00.032848+0000","last_peered":"2024-06-02T15:40:00.032848+0000","last_clean":"2024-06-02T15:40:00.032848+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:40:00.032848+0000","last_undegraded":"2024-06-02T15:40:00.032848+0000","last_fullsized":"2024-06-02T15:40:00.032848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030517+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:00.030517+0000","last_peered":"2024-06-02T15:40:00.030517+0000","last_clean":"2024-06-02T15:40:00.030517+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:00.030517+0000","last_undegraded":"2024-06-02T15:40:00.030517+0000","last_fullsized":"2024-06-02T15:40:00.030517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031614+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:00.031614+0000","last_peered":"2024-06-02T15:40:00.031614+0000","last_clean":"2024-06-02T15:40:00.031614+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:00.031614+0000","last_undegraded":"2024-06-02T15:40:00.031614+0000","last_fullsized":"2024-06-02T15:40:00.031614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.028585+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:40:00.028585+0000","last_peered":"2024-06-02T15:40:00.028585+0000","last_clean":"2024-06-02T15:40:00.028585+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:40:00.028585+0000","last_undegraded":"2024-06-02T15:40:00.028585+0000","last_fullsized":"2024-06-02T15:40:00.028585+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031473+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:00.031473+0000","last_peered":"2024-06-02T15:40:00.031473+0000","last_clean":"2024-06-02T15:40:00.031473+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:00.031473+0000","last_undegraded":"2024-06-02T15:40:00.031473+0000","last_fullsized":"2024-06-02T15:40:00.031473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027595+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:00.027595+0000","last_peered":"2024-06-02T15:40:00.027595+0000","last_clean":"2024-06-02T15:40:00.027595+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:00.027595+0000","last_undegraded":"2024-06-02T15:40:00.027595+0000","last_fullsized":"2024-06-02T15:40:00.027595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031962+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:40:00.031962+0000","last_peered":"2024-06-02T15:40:00.031962+0000","last_clean":"2024-06-02T15:40:00.031962+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:40:00.031962+0000","last_undegraded":"2024-06-02T15:40:00.031962+0000","last_fullsized":"2024-06-02T15:40:00.031962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036615+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:00.036615+0000","last_peered":"2024-06-02T15:40:00.036615+0000","last_clean":"2024-06-02T15:40:00.036615+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:00.036615+0000","last_undegraded":"2024-06-02T15:40:00.036615+0000","last_fullsized":"2024-06-02T15:40:00.036615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032099+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:00.032099+0000","last_peered":"2024-06-02T15:40:00.032099+0000","last_clean":"2024-06-02T15:40:00.032099+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:00.032099+0000","last_undegraded":"2024-06-02T15:40:00.032099+0000","last_fullsized":"2024-06-02T15:40:00.032099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029548+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:40:00.029548+0000","last_peered":"2024-06-02T15:40:00.029548+0000","last_clean":"2024-06-02T15:40:00.029548+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:40:00.029548+0000","last_undegraded":"2024-06-02T15:40:00.029548+0000","last_fullsized":"2024-06-02T15:40:00.029548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032297+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:00.032297+0000","last_peered":"2024-06-02T15:40:00.032297+0000","last_clean":"2024-06-02T15:40:00.032297+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:00.032297+0000","last_undegraded":"2024-06-02T15:40:00.032297+0000","last_fullsized":"2024-06-02T15:40:00.032297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029712+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:40:00.029712+0000","last_peered":"2024-06-02T15:40:00.029712+0000","last_clean":"2024-06-02T15:40:00.029712+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:40:00.029712+0000","last_undegraded":"2024-06-02T15:40:00.029712+0000","last_fullsized":"2024-06-02T15:40:00.029712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033286+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:40:00.033286+0000","last_peered":"2024-06-02T15:40:00.033286+0000","last_clean":"2024-06-02T15:40:00.033286+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:40:00.033286+0000","last_undegraded":"2024-06-02T15:40:00.033286+0000","last_fullsized":"2024-06-02T15:40:00.033286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031434+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:00.031434+0000","last_peered":"2024-06-02T15:40:00.031434+0000","last_clean":"2024-06-02T15:40:00.031434+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:00.031434+0000","last_undegraded":"2024-06-02T15:40:00.031434+0000","last_fullsized":"2024-06-02T15:40:00.031434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032138+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:00.032138+0000","last_peered":"2024-06-02T15:40:00.032138+0000","last_clean":"2024-06-02T15:40:00.032138+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:00.032138+0000","last_undegraded":"2024-06-02T15:40:00.032138+0000","last_fullsized":"2024-06-02T15:40:00.032138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027001+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:40:00.027001+0000","last_peered":"2024-06-02T15:40:00.027001+0000","last_clean":"2024-06-02T15:40:00.027001+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:40:00.027001+0000","last_undegraded":"2024-06-02T15:40:00.027001+0000","last_fullsized":"2024-06-02T15:40:00.027001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027767+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:00.027767+0000","last_peered":"2024-06-02T15:40:00.027767+0000","last_clean":"2024-06-02T15:40:00.027767+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:00.027767+0000","last_undegraded":"2024-06-02T15:40:00.027767+0000","last_fullsized":"2024-06-02T15:40:00.027767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031020+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:00.031020+0000","last_peered":"2024-06-02T15:40:00.031020+0000","last_clean":"2024-06-02T15:40:00.031020+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:00.031020+0000","last_undegraded":"2024-06-02T15:40:00.031020+0000","last_fullsized":"2024-06-02T15:40:00.031020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027988+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:40:00.027988+0000","last_peered":"2024-06-02T15:40:00.027988+0000","last_clean":"2024-06-02T15:40:00.027988+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:40:00.027988+0000","last_undegraded":"2024-06-02T15:40:00.027988+0000","last_fullsized":"2024-06-02T15:40:00.027988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034227+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:00.034227+0000","last_peered":"2024-06-02T15:40:00.034227+0000","last_clean":"2024-06-02T15:40:00.034227+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:00.034227+0000","last_undegraded":"2024-06-02T15:40:00.034227+0000","last_fullsized":"2024-06-02T15:40:00.034227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024284+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:00.024284+0000","last_peered":"2024-06-02T15:40:00.024284+0000","last_clean":"2024-06-02T15:40:00.024284+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:00.024284+0000","last_undegraded":"2024-06-02T15:40:00.024284+0000","last_fullsized":"2024-06-02T15:40:00.024284+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027395+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:40:00.027395+0000","last_peered":"2024-06-02T15:40:00.027395+0000","last_clean":"2024-06-02T15:40:00.027395+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:40:00.027395+0000","last_undegraded":"2024-06-02T15:40:00.027395+0000","last_fullsized":"2024-06-02T15:40:00.027395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023654+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:00.023654+0000","last_peered":"2024-06-02T15:40:00.023654+0000","last_clean":"2024-06-02T15:40:00.023654+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:00.023654+0000","last_undegraded":"2024-06-02T15:40:00.023654+0000","last_fullsized":"2024-06-02T15:40:00.023654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034978+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:00.034978+0000","last_peered":"2024-06-02T15:40:00.034978+0000","last_clean":"2024-06-02T15:40:00.034978+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:00.034978+0000","last_undegraded":"2024-06-02T15:40:00.034978+0000","last_fullsized":"2024-06-02T15:40:00.034978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027634+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:40:00.027634+0000","last_peered":"2024-06-02T15:40:00.027634+0000","last_clean":"2024-06-02T15:40:00.027634+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:40:00.027634+0000","last_undegraded":"2024-06-02T15:40:00.027634+0000","last_fullsized":"2024-06-02T15:40:00.027634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029555+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:40:00.029555+0000","last_peered":"2024-06-02T15:40:00.029555+0000","last_clean":"2024-06-02T15:40:00.029555+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:40:00.029555+0000","last_undegraded":"2024-06-02T15:40:00.029555+0000","last_fullsized":"2024-06-02T15:40:00.029555+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030706+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:40:00.030706+0000","last_peered":"2024-06-02T15:40:00.030706+0000","last_clean":"2024-06-02T15:40:00.030706+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:40:00.030706+0000","last_undegraded":"2024-06-02T15:40:00.030706+0000","last_fullsized":"2024-06-02T15:40:00.030706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030392+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:40:00.030392+0000","last_peered":"2024-06-02T15:40:00.030392+0000","last_clean":"2024-06-02T15:40:00.030392+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:40:00.030392+0000","last_undegraded":"2024-06-02T15:40:00.030392+0000","last_fullsized":"2024-06-02T15:40:00.030392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026736+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:40:00.026736+0000","last_peered":"2024-06-02T15:40:00.026736+0000","last_clean":"2024-06-02T15:40:00.026736+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:40:00.026736+0000","last_undegraded":"2024-06-02T15:40:00.026736+0000","last_fullsized":"2024-06-02T15:40:00.026736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038178+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:40:00.038178+0000","last_peered":"2024-06-02T15:40:00.038178+0000","last_clean":"2024-06-02T15:40:00.038178+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:40:00.038178+0000","last_undegraded":"2024-06-02T15:40:00.038178+0000","last_fullsized":"2024-06-02T15:40:00.038178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030866+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:40:00.030866+0000","last_peered":"2024-06-02T15:40:00.030866+0000","last_clean":"2024-06-02T15:40:00.030866+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:40:00.030866+0000","last_undegraded":"2024-06-02T15:40:00.030866+0000","last_fullsized":"2024-06-02T15:40:00.030866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031038+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:40:00.031038+0000","last_peered":"2024-06-02T15:40:00.031038+0000","last_clean":"2024-06-02T15:40:00.031038+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:40:00.031038+0000","last_undegraded":"2024-06-02T15:40:00.031038+0000","last_fullsized":"2024-06-02T15:40:00.031038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033098+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:40:00.033098+0000","last_peered":"2024-06-02T15:40:00.033098+0000","last_clean":"2024-06-02T15:40:00.033098+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:40:00.033098+0000","last_undegraded":"2024-06-02T15:40:00.033098+0000","last_fullsized":"2024-06-02T15:40:00.033098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023785+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:40:00.023785+0000","last_peered":"2024-06-02T15:40:00.023785+0000","last_clean":"2024-06-02T15:40:00.023785+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:40:00.023785+0000","last_undegraded":"2024-06-02T15:40:00.023785+0000","last_fullsized":"2024-06-02T15:40:00.023785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032192+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:40:00.032192+0000","last_peered":"2024-06-02T15:40:00.032192+0000","last_clean":"2024-06-02T15:40:00.032192+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:40:00.032192+0000","last_undegraded":"2024-06-02T15:40:00.032192+0000","last_fullsized":"2024-06-02T15:40:00.032192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027945+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:40:00.027945+0000","last_peered":"2024-06-02T15:40:00.027945+0000","last_clean":"2024-06-02T15:40:00.027945+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:40:00.027945+0000","last_undegraded":"2024-06-02T15:40:00.027945+0000","last_fullsized":"2024-06-02T15:40:00.027945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034023+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:40:00.034023+0000","last_peered":"2024-06-02T15:40:00.034023+0000","last_clean":"2024-06-02T15:40:00.034023+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:40:00.034023+0000","last_undegraded":"2024-06-02T15:40:00.034023+0000","last_fullsized":"2024-06-02T15:40:00.034023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030141+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:40:00.030141+0000","last_peered":"2024-06-02T15:40:00.030141+0000","last_clean":"2024-06-02T15:40:00.030141+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:40:00.030141+0000","last_undegraded":"2024-06-02T15:40:00.030141+0000","last_fullsized":"2024-06-02T15:40:00.030141+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68452,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":240,"num_read_kb":195,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1169879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037811,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234042,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1114034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":577542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":515223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":582673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1646592,"data_stored":1105717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:40:16.621 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-02T15:40:16.621 INFO:tasks.cephadm:Setting up client nodes... 2024-06-02T15:40:16.622 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-02T15:40:16.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:16 smithi090 ceph-mon[31595]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:17.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:16 smithi120 ceph-mon[36451]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:17.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:16 smithi135 ceph-mon[36839]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:17.114 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:18.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:18 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:18 smithi090 ceph-mon[31595]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:18 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:18 smithi120 ceph-mon[36451]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:18 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:18 smithi135 ceph-mon[36839]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 -- 172.21.15.90:0/3061827042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1047d0 msgr2=0x7f3b8c104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 --2- 172.21.15.90:0/3061827042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c104c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b780099c0 tx=0x7f3b78009cd0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 -- 172.21.15.90:0/3061827042 shutdown_connections 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 --2- 172.21.15.90:0/3061827042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c105170 0x7f3b8c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 --2- 172.21.15.90:0/3061827042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 --2- 172.21.15.90:0/3061827042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c104c30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 -- 172.21.15.90:0/3061827042 >> 172.21.15.90:0/3061827042 conn(0x7f3b8c0feb90 msgr2=0x7f3b8c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 -- 172.21.15.90:0/3061827042 shutdown_connections 2024-06-02T15:40:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.365+0000 7f3b94200700 1 -- 172.21.15.90:0/3061827042 wait complete. 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.366+0000 7f3b94200700 1 Processor -- start 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.366+0000 7f3b94200700 1 -- start start 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 0x7f3b8c075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10f050 con 0x7f3b8c1035d0 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10eed0 con 0x7f3b8c105170 2024-06-02T15:40:19.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b94200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10f1d0 con 0x7f3b8c1047d0 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b91800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b91800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57638/0 (socket says 172.21.15.90:57638) 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b91800700 1 -- 172.21.15.90:0/2173116151 learned_addr learned my addr 172.21.15.90:0/2173116151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b90e00700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.367+0000 7f3b92200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 0x7f3b8c075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 -- 172.21.15.90:0/2173116151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c1047d0 msgr2=0x7f3b8c075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 -- 172.21.15.90:0/2173116151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 msgr2=0x7f3b8c075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 0x7f3b8c075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 -- 172.21.15.90:0/2173116151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c076720 con 0x7f3b8c1035d0 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b92200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 0x7f3b8c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:19.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b91800700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000d980 tx=0x7f3b8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80017030 con 0x7f3b8c1035d0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1a3770 con 0x7f3b8c1035d0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b800044d0 con 0x7f3b8c1035d0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b80003970 con 0x7f3b8c1035d0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.368+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1a3cb0 con 0x7f3b8c1035d0 2024-06-02T15:40:19.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.370+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c051a80 con 0x7f3b8c1035d0 2024-06-02T15:40:19.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.372+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b8001a070 con 0x7f3b8c1035d0 2024-06-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.374+0000 7f3b8a000700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 0x7f3b740b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.374+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3b800e1fa0 con 0x7f3b8c1035d0 2024-06-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.374+0000 7f3b90e00700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 0x7f3b740b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:19.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.375+0000 7f3b90e00700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 0x7f3b740b4ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009990 tx=0x7f3b780096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.379+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b800a9750 con 0x7f3b8c1035d0 2024-06-02T15:40:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:40:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:40:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:19.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.709+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3b8c02e600 con 0x7f3b8c1035d0 2024-06-02T15:40:19.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.716+0000 7f3b8a000700 1 -- 172.21.15.90:0/2173116151 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3b800a9050 con 0x7f3b8c1035d0 2024-06-02T15:40:19.715 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-06-02T15:40:19.715 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBjklxm88J2KhAAXMYNbnFMPyBaBcNu/AcR+A== 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 msgr2=0x7f3b740b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 0x7f3b740b4ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009990 tx=0x7f3b780096b0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 msgr2=0x7f3b8c0772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000d980 tx=0x7f3b8000dc90 comp rx=0 tx=0).stop 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 shutdown_connections 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3b740b2a30 0x7f3b740b4ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b8c105170 0x7f3b8c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c1035d0 0x7f3b8c0772e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 --2- 172.21.15.90:0/2173116151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b8c1047d0 0x7f3b8c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:19.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.719+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 >> 172.21.15.90:0/2173116151 conn(0x7f3b8c0feb90 msgr2=0x7f3b8c107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:19.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.720+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 shutdown_connections 2024-06-02T15:40:19.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:19.720+0000 7f3b94200700 1 -- 172.21.15.90:0/2173116151 wait complete. 2024-06-02T15:40:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:40:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:40:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:40:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:40:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:40:20.514 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:40:20.515 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-02T15:40:20.515 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-02T15:40:20.634 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-02T15:40:20.634 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-02T15:40:20.634 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-02T15:40:20.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T15:40:20.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T15:40:20.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:20 smithi090 ceph-mon[31595]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-02T15:40:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T15:40:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T15:40:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:20 smithi120 ceph-mon[36451]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-02T15:40:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T15:40:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2173116151' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T15:40:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:20 smithi135 ceph-mon[36839]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-06-02T15:40:21.122 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:22 smithi120 ceph-mon[36451]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:40:23.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:22 smithi135 ceph-mon[36839]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:40:23.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:22 smithi090 ceph-mon[31595]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 -- 172.21.15.90:0/217846308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c075770 msgr2=0x7fa73c075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 --2- 172.21.15.90:0/217846308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c075770 0x7fa73c075b50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa730009990 tx=0x7fa730009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 -- 172.21.15.90:0/217846308 shutdown_connections 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 --2- 172.21.15.90:0/217846308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c114710 0x7fa73c116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 --2- 172.21.15.90:0/217846308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c075770 0x7fa73c075b50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 --2- 172.21.15.90:0/217846308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c076090 0x7fa73c1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 -- 172.21.15.90:0/217846308 >> 172.21.15.90:0/217846308 conn(0x7fa73c07a6a0 msgr2=0x7fa73c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 -- 172.21.15.90:0/217846308 shutdown_connections 2024-06-02T15:40:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.254+0000 7fa743c00700 1 -- 172.21.15.90:0/217846308 wait complete. 2024-06-02T15:40:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 Processor -- start 2024-06-02T15:40:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 -- start start 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 0x7fa73c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c114710 0x7fa73c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c119d60 con 0x7fa73c075770 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c119be0 con 0x7fa73c076090 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.256+0000 7fa743c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c119ee0 con 0x7fa73c114710 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa741200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 0x7fa73c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa741200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 0x7fa73c1a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47182/0 (socket says 172.21.15.90:47182) 2024-06-02T15:40:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa741200700 1 -- 172.21.15.90:0/1348761609 learned_addr learned my addr 172.21.15.90:0/1348761609 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39960/0 (socket says 172.21.15.90:39960) 2024-06-02T15:40:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa741c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c114710 0x7fa73c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 -- 172.21.15.90:0/1348761609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c114710 msgr2=0x7fa73c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c114710 0x7fa73c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 -- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 msgr2=0x7fa73c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 0x7fa73c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.257+0000 7fa73be00700 1 -- 172.21.15.90:0/1348761609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c1a52c0 con 0x7fa73c076090 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa73be00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9f0 tx=0x7fa72800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728004220 con 0x7fa73c076090 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7280043c0 con 0x7fa73c076090 2024-06-02T15:40:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c1a55b0 con 0x7fa73c076090 2024-06-02T15:40:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728005020 con 0x7fa73c076090 2024-06-02T15:40:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.258+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c1ace30 con 0x7fa73c076090 2024-06-02T15:40:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.259+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73c06a230 con 0x7fa73c076090 2024-06-02T15:40:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.261+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa728004750 con 0x7fa73c076090 2024-06-02T15:40:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.262+0000 7fa739600700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 0x7fa7240b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.263+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7280e26c0 con 0x7fa73c076090 2024-06-02T15:40:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.263+0000 7fa741200700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 0x7fa7240b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.263+0000 7fa741200700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 0x7fa7240b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa73000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:23.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.267+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7280a9e70 con 0x7fa73c076090 2024-06-02T15:40:23.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.569+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa73c1ad060 con 0x7fa73c076090 2024-06-02T15:40:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.575+0000 7fa739600700 1 -- 172.21.15.90:0/1348761609 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288643 (secure 0 0 0) 0x7fa7280a9770 con 0x7fa73c076090 2024-06-02T15:40:23.580 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 msgr2=0x7fa7240b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 0x7fa7240b4e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa730006010 tx=0x7fa73000b280 comp rx=0 tx=0).stop 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 msgr2=0x7fa73c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa72800e9f0 tx=0x7fa72800ed00 comp rx=0 tx=0).stop 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 shutdown_connections 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7240b2990 0x7fa7240b4e50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa73c114710 0x7fa73c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa73c075770 0x7fa73c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 --2- 172.21.15.90:0/1348761609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa73c076090 0x7fa73c1a07f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:23.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.582+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 >> 172.21.15.90:0/1348761609 conn(0x7fa73c07a6a0 msgr2=0x7fa73c1135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:23.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.583+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 shutdown_connections 2024-06-02T15:40:23.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:23.583+0000 7fa743c00700 1 -- 172.21.15.90:0/1348761609 wait complete. 2024-06-02T15:40:23.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:23 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1348761609' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T15:40:24.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:23 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1348761609' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T15:40:24.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:23 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1348761609' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T15:40:24.521 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":436264481},{"type":"v1","addr":"172.21.15.90:6801","nonce":436264481}]},"active_addr":"172.21.15.90:6801/436264481","active_change":"2024-06-02T15:29:57.532295+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1527619074}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":742359897}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":4175344633}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3327243880}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1623155276}]}]} 2024-06-02T15:40:24.526 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-02T15:40:24.527 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-02T15:40:24.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:40:24.976 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:25.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:24 smithi120 ceph-mon[36451]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:25.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:24 smithi135 ceph-mon[36839]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:24 smithi090 ceph-mon[31595]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:27.000 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:26 smithi090 ceph-mon[31595]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:27.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:26 smithi135 ceph-mon[36839]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:26 smithi120 ceph-mon[36451]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:27.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 -- 172.21.15.90:0/29597714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba0076090 msgr2=0x7f8ba010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:27.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 --2- 172.21.15.90:0/29597714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba0076090 0x7f8ba010fa00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009990 tx=0x7f8b8c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 -- 172.21.15.90:0/29597714 shutdown_connections 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 --2- 172.21.15.90:0/29597714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba010ff40 0x7f8ba0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 --2- 172.21.15.90:0/29597714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 --2- 172.21.15.90:0/29597714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba0076090 0x7f8ba010fa00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 -- 172.21.15.90:0/29597714 >> 172.21.15.90:0/29597714 conn(0x7f8ba007a6a0 msgr2=0x7f8ba007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 -- 172.21.15.90:0/29597714 shutdown_connections 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.278+0000 7f8ba8600700 1 -- 172.21.15.90:0/29597714 wait complete. 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.279+0000 7f8ba8600700 1 Processor -- start 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 -- start start 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba0076090 0x7f8ba019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:27.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba010ff40 0x7f8ba01a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0115960 con 0x7f8ba010ff40 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba01157e0 con 0x7f8ba0076090 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba8600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba0115ae0 con 0x7f8ba0075770 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48046/0 (socket says 172.21.15.90:48046) 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba5c00700 1 -- 172.21.15.90:0/3151649424 learned_addr learned my addr 172.21.15.90:0/3151649424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba5200700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba0076090 0x7f8ba019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.280+0000 7f8ba6600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba010ff40 0x7f8ba01a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 -- 172.21.15.90:0/3151649424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba0076090 msgr2=0x7f8ba019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba0076090 0x7f8ba019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 -- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba010ff40 msgr2=0x7f8ba01a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba010ff40 0x7f8ba01a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 -- 172.21.15.90:0/3151649424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01a0e90 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8ba5c00700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400ef30 tx=0x7f8b9400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.281+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b940092e0 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.282+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b94004bb0 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.282+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba01a1120 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.282+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94004020 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.282+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba01a8a00 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.283+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba006a230 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.285+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8b9401b030 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.287+0000 7f8b9e000700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.287+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8b940e2cb0 con 0x7f8ba0075770 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.287+0000 7f8ba5200700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.288+0000 7f8ba5200700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 0x7f8b880b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0103110 tx=0x7f8b8c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:27.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.289+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b94022020 con 0x7f8ba0075770 2024-06-02T15:40:27.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.520+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ba002e560 con 0x7f8ba0075770 2024-06-02T15:40:27.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.523+0000 7f8b9e000700 1 -- 172.21.15.90:0/3151649424 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7f8b94010040 con 0x7f8ba0075770 2024-06-02T15:40:27.521 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:27.522 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:40:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 msgr2=0x7f8b880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 0x7f8b880b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0103110 tx=0x7f8b8c00b260 comp rx=0 tx=0).stop 2024-06-02T15:40:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 msgr2=0x7f8ba019bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400ef30 tx=0x7f8b9400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 shutdown_connections 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8b880b2910 0x7f8b880b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba010ff40 0x7f8ba01a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ba0075770 0x7f8ba019bed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 --2- 172.21.15.90:0/3151649424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8ba0076090 0x7f8ba019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 >> 172.21.15.90:0/3151649424 conn(0x7f8ba007a6a0 msgr2=0x7f8ba01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 shutdown_connections 2024-06-02T15:40:27.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:27.526+0000 7f8ba8600700 1 -- 172.21.15.90:0/3151649424 wait complete. 2024-06-02T15:40:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:27 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3151649424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:28.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:27 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3151649424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:28.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:27 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3151649424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:28.522 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-02T15:40:28.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:40:28.983 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:29.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:28 smithi090 ceph-mon[31595]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:28 smithi120 ceph-mon[36451]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:28 smithi135 ceph-mon[36839]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:30.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:30 smithi090 ceph-mon[31595]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:31.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:30 smithi120 ceph-mon[36451]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:31.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:30 smithi135 ceph-mon[36839]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:31.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 -- 172.21.15.90:0/3059141633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa98810b760 msgr2=0x7fa98810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:31.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 --2- 172.21.15.90:0/3059141633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa98810b760 0x7fa98810bb40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa978009990 tx=0x7fa978009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 -- 172.21.15.90:0/3059141633 shutdown_connections 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 --2- 172.21.15.90:0/3059141633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988106120 0x7fa988112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 --2- 172.21.15.90:0/3059141633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa98810b760 0x7fa98810bb40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 --2- 172.21.15.90:0/3059141633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa988105760 0x7fa988105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 -- 172.21.15.90:0/3059141633 >> 172.21.15.90:0/3059141633 conn(0x7fa988101250 msgr2=0x7fa988103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 -- 172.21.15.90:0/3059141633 shutdown_connections 2024-06-02T15:40:31.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.663+0000 7fa98ec00700 1 -- 172.21.15.90:0/3059141633 wait complete. 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.665+0000 7fa98ec00700 1 Processor -- start 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.665+0000 7fa98ec00700 1 -- start start 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.665+0000 7fa98ec00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988105760 0x7fa98819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.665+0000 7fa98ec00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa98ec00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa98810b760 0x7fa9881a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa98ec00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa988115940 con 0x7fa988106120 2024-06-02T15:40:31.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa98ec00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881157c0 con 0x7fa98810b760 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa98ec00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa988115ac0 con 0x7fa988105760 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42818/0 (socket says 172.21.15.90:42818) 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 -- 172.21.15.90:0/4244263337 learned_addr learned my addr 172.21.15.90:0/4244263337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa987e00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988105760 0x7fa98819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:31.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa98cc00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa98810b760 0x7fa9881a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 -- 172.21.15.90:0/4244263337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988105760 msgr2=0x7fa98819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988105760 0x7fa98819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 -- 172.21.15.90:0/4244263337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa98810b760 msgr2=0x7fa9881a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa98810b760 0x7fa9881a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.666+0000 7fa97fe00700 1 -- 172.21.15.90:0/4244263337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9881a0e00 con 0x7fa988106120 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa97fe00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97000e9a0 tx=0x7fa97000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97000ca10 con 0x7fa988106120 2024-06-02T15:40:31.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa97000cbb0 con 0x7fa988106120 2024-06-02T15:40:31.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970018680 con 0x7fa988106120 2024-06-02T15:40:31.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881a10f0 con 0x7fa988106120 2024-06-02T15:40:31.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.667+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881a8970 con 0x7fa988106120 2024-06-02T15:40:31.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.668+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98806a230 con 0x7fa988106120 2024-06-02T15:40:31.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.672+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa970010430 con 0x7fa988106120 2024-06-02T15:40:31.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.674+0000 7fa985600700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 0x7fa96c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:31.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.674+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa9700e2c60 con 0x7fa988106120 2024-06-02T15:40:31.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.674+0000 7fa987e00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 0x7fa96c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:31.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.675+0000 7fa987e00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 0x7fa96c0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098e0 tx=0x7fa978005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:31.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.677+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9700aa410 con 0x7fa988106120 2024-06-02T15:40:31.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.915+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa98802e720 con 0x7fa988106120 2024-06-02T15:40:31.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.920+0000 7fa985600700 1 -- 172.21.15.90:0/4244263337 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7fa9700a9d10 con 0x7fa988106120 2024-06-02T15:40:31.919 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:40:31.919 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:40:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.923+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 msgr2=0x7fa96c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.923+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 0x7fa96c0b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098e0 tx=0x7fa978005960 comp rx=0 tx=0).stop 2024-06-02T15:40:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 msgr2=0x7fa98819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97000e9a0 tx=0x7fa97000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 shutdown_connections 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa96c0b2910 0x7fa96c0b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa98810b760 0x7fa9881a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa988105760 0x7fa98819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 --2- 172.21.15.90:0/4244263337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa988106120 0x7fa98819c330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 >> 172.21.15.90:0/4244263337 conn(0x7fa988101250 msgr2=0x7fa9881018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 shutdown_connections 2024-06-02T15:40:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:31.924+0000 7fa98ec00700 1 -- 172.21.15.90:0/4244263337 wait complete. 2024-06-02T15:40:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:32 smithi090 ceph-mon[31595]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:32 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4244263337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:33.181 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-02T15:40:33.182 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-02T15:40:33.182 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-02T15:40:33.183 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-02T15:40:33.183 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-02T15:40:33.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-02T15:40:33.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-02T15:40:33.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-02T15:40:33.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-02T15:40:33.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-02T15:40:33.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-02T15:40:33.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-02T15:40:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:32 smithi120 ceph-mon[36451]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:32 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4244263337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:32 smithi135 ceph-mon[36839]: pgmap v397: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:32 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4244263337' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:40:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:35.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:34 smithi090 ceph-mon[31595]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:34 smithi120 ceph-mon[36451]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:34 smithi135 ceph-mon[36839]: pgmap v398: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.244 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.244 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.252 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.257 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:36.283 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:40:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:36 smithi090 ceph-mon[31595]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:36 smithi120 ceph-mon[36451]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:36 smithi135 ceph-mon[36839]: pgmap v399: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:39.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:38 smithi090 ceph-mon[31595]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:38 smithi120 ceph-mon[36451]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:38 smithi135 ceph-mon[36839]: pgmap v400: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:40.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:40 smithi090 ceph-mon[31595]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:41.119 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:40 smithi135 ceph-mon[36839]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:40 smithi120 ceph-mon[36451]: pgmap v401: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:43.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:43 smithi090 ceph-mon[31595]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:43 smithi120 ceph-mon[36451]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:43 smithi135 ceph-mon[36839]: pgmap v402: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:45 smithi120 ceph-mon[36451]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:45 smithi135 ceph-mon[36839]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:45.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:45 smithi090 ceph-mon[31595]: pgmap v403: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:47.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:47 smithi090 ceph-mon[31595]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:47 smithi120 ceph-mon[36451]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:47 smithi135 ceph-mon[36839]: pgmap v404: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:49.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:49 smithi090 ceph-mon[31595]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:49 smithi120 ceph-mon[36451]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:49 smithi135 ceph-mon[36839]: pgmap v405: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 -- 172.21.15.90:0/2866055867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c104630 msgr2=0x7fd37c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 --2- 172.21.15.90:0/2866055867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c104630 0x7fd37c104a10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd370009990 tx=0x7fd370009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 -- 172.21.15.90:0/2866055867 shutdown_connections 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 --2- 172.21.15.90:0/2866055867 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c10da60 0x7fd37c10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 --2- 172.21.15.90:0/2866055867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c104630 0x7fd37c104a10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 --2- 172.21.15.90:0/2866055867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104f50 0x7fd37c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 -- 172.21.15.90:0/2866055867 >> 172.21.15.90:0/2866055867 conn(0x7fd37c0fde60 msgr2=0x7fd37c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 -- 172.21.15.90:0/2866055867 shutdown_connections 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.835+0000 7fd382e00700 1 -- 172.21.15.90:0/2866055867 wait complete. 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.837+0000 7fd382e00700 1 Processor -- start 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.837+0000 7fd382e00700 1 -- start start 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.837+0000 7fd382e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 0x7fd37c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.837+0000 7fd382e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c104f50 0x7fd37c1027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd382e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c10da60 0x7fd37c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd382e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c113670 con 0x7fd37c104630 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd382e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1134f0 con 0x7fd37c104f50 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd382e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1137f0 con 0x7fd37c10da60 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd381a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 0x7fd37c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd381a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 0x7fd37c102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35706/0 (socket says 172.21.15.90:35706) 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd381a00700 1 -- 172.21.15.90:0/3701864843 learned_addr learned my addr 172.21.15.90:0/3701864843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd382400700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c10da60 0x7fd37c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.838+0000 7fd381000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c104f50 0x7fd37c1027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 -- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c10da60 msgr2=0x7fd37c102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c10da60 0x7fd37c102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 -- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 msgr2=0x7fd37c102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 0x7fd37c102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c1a8070 con 0x7fd37c104f50 2024-06-02T15:40:49.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381a00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd37c104630 0x7fd37c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd382400700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd37c10da60 0x7fd37c102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd381000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c104f50 0x7fd37c1027d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd36800e9c0 tx=0x7fd36800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36800c9c0 con 0x7fd37c104f50 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36800cb60 con 0x7fd37c104f50 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368018420 con 0x7fd37c104f50 2024-06-02T15:40:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.839+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c1a8360 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.840+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c1a8850 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.840+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd37c063c50 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.844+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd368010430 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.845+0000 7fd37a000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3640b2990 0x7fd3640b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd3680e30e0 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd381a00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3640b2990 0x7fd3640b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd37a000700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] conn(0x7fd3640bdc70 0x7fd3640c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd368028ea0 con 0x7fd3640bdc70 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd3680e3490 con 0x7fd37c104f50 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd382400700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] conn(0x7fd3640bdc70 0x7fd3640c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.846+0000 7fd381a00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3640b2990 0x7fd3640b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd370006010 tx=0x7fd370009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.847+0000 7fd382400700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] conn(0x7fd3640bdc70 0x7fd3640c0110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:49.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.848+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== osd.8 v2:172.21.15.135:6816/1073792121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd368028ea0 con 0x7fd3640bdc70 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.866+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 --> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd37c064120 con 0x7fd3640bdc70 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.866+0000 7fd37a000700 1 -- 172.21.15.90:0/3701864843 <== osd.8 v2:172.21.15.135:6816/1073792121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd37c064120 con 0x7fd3640bdc70 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] conn(0x7fd3640bdc70 msgr2=0x7fd3640c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] conn(0x7fd3640bdc70 0x7fd3640c0110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3640b2990 msgr2=0x7fd3640b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd3640b2990 0x7fd3640b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd370006010 tx=0x7fd370009510 comp rx=0 tx=0).stop 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c104f50 msgr2=0x7fd37c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 --2- 172.21.15.90:0/3701864843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd37c104f50 0x7fd37c1027d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd36800e9c0 tx=0x7fd36800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382400700 1 -- 172.21.15.90:0/3701864843 reap_dead start 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 shutdown_connections 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.867+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 >> 172.21.15.90:0/3701864843 conn(0x7fd37c0fde60 msgr2=0x7fd37c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.868+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 shutdown_connections 2024-06-02T15:40:49.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:49.868+0000 7fd382e00700 1 -- 172.21.15.90:0/3701864843 wait complete. 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 -- 172.21.15.90:0/1461641302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8406c860 msgr2=0x7fea84108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 --2- 172.21.15.90:0/1461641302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8406c860 0x7fea84108180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea740098f0 tx=0x7fea74009c00 comp rx=0 tx=0).stop 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 -- 172.21.15.90:0/1461641302 shutdown_connections 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 --2- 172.21.15.90:0/1461641302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea84108860 0x7fea84110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 --2- 172.21.15.90:0/1461641302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea8406c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 --2- 172.21.15.90:0/1461641302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8406c860 0x7fea84108180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 -- 172.21.15.90:0/1461641302 >> 172.21.15.90:0/1461641302 conn(0x7fea84078490 msgr2=0x7fea840788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 -- 172.21.15.90:0/1461641302 shutdown_connections 2024-06-02T15:40:50.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.628+0000 7fea8c800700 1 -- 172.21.15.90:0/1461641302 wait complete. 2024-06-02T15:40:50.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.629+0000 7fea8c800700 1 Processor -- start 2024-06-02T15:40:50.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.629+0000 7fea8c800700 1 -- start start 2024-06-02T15:40:50.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea841025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 0x7fea84102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea84108860 0x7fea84103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea841137a0 con 0x7fea8406c860 2024-06-02T15:40:50.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea84113620 con 0x7fea84108860 2024-06-02T15:40:50.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8c800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea84113920 con 0x7fea8406beb0 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea89400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 0x7fea84102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea89e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea841025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea89e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea841025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39102/0 (socket says 172.21.15.90:39102) 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea89400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 0x7fea84102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35732/0 (socket says 172.21.15.90:35732) 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea89e00700 1 -- 172.21.15.90:0/2193222674 learned_addr learned my addr 172.21.15.90:0/2193222674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.630+0000 7fea8a800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea84108860 0x7fea84103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 -- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea84108860 msgr2=0x7fea84103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea84108860 0x7fea84103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 -- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 msgr2=0x7fea84102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 0x7fea84102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea841a8240 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89400700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8406c860 0x7fea84102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea89e00700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea841025a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e9c0 tx=0x7fea7800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.631+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7800cbd0 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.632+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea841a8530 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.632+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7800ee60 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.632+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780055c0 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.632+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea841a89f0 con 0x7fea8406beb0 2024-06-02T15:40:50.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.633+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea84063c50 con 0x7fea8406beb0 2024-06-02T15:40:50.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.635+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fea78004750 con 0x7fea8406beb0 2024-06-02T15:40:50.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.636+0000 7fea7ea00700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fea6c0b2910 0x7fea6c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fea780e2e40 con 0x7fea8406beb0 2024-06-02T15:40:50.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea89400700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fea6c0b2910 0x7fea6c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea7ea00700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] conn(0x7fea6c0bdbf0 0x7fea6c0c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea7800be50 con 0x7fea6c0bdbf0 2024-06-02T15:40:50.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fea780e31f0 con 0x7fea8406beb0 2024-06-02T15:40:50.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.637+0000 7fea8a800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] conn(0x7fea6c0bdbf0 0x7fea6c0c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.638+0000 7fea89400700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fea6c0b2910 0x7fea6c0b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.638+0000 7fea8a800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] conn(0x7fea6c0bdbf0 0x7fea6c0c0090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.639+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== osd.7 v2:172.21.15.120:6816/1944245064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea7800be50 con 0x7fea6c0bdbf0 2024-06-02T15:40:50.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.657+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 --> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea840640d0 con 0x7fea6c0bdbf0 2024-06-02T15:40:50.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.658+0000 7fea7ea00700 1 -- 172.21.15.90:0/2193222674 <== osd.7 v2:172.21.15.120:6816/1944245064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea840640d0 con 0x7fea6c0bdbf0 2024-06-02T15:40:50.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.658+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] conn(0x7fea6c0bdbf0 msgr2=0x7fea6c0c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.658+0000 7fea8c800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] conn(0x7fea6c0bdbf0 0x7fea6c0c0090 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fea6c0b2910 msgr2=0x7fea6c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fea6c0b2910 0x7fea6c0b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea74009fd0 tx=0x7fea74019040 comp rx=0 tx=0).stop 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 msgr2=0x7fea841025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 --2- 172.21.15.90:0/2193222674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea8406beb0 0x7fea841025a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea7800e9c0 tx=0x7fea7800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8a800700 1 -- 172.21.15.90:0/2193222674 reap_dead start 2024-06-02T15:40:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 shutdown_connections 2024-06-02T15:40:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 >> 172.21.15.90:0/2193222674 conn(0x7fea84078490 msgr2=0x7fea84100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 shutdown_connections 2024-06-02T15:40:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.659+0000 7fea8c800700 1 -- 172.21.15.90:0/2193222674 wait complete. 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 -- 172.21.15.90:0/636929797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb01095b0 msgr2=0x7f1cb0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 --2- 172.21.15.90:0/636929797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb01095b0 0x7f1cb0109990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400b0a0 tx=0x7f1ca400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 -- 172.21.15.90:0/636929797 shutdown_connections 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 --2- 172.21.15.90:0/636929797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb006bbe0 0x7f1cb01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 --2- 172.21.15.90:0/636929797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb01095b0 0x7f1cb0109990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 --2- 172.21.15.90:0/636929797 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cb006b220 0x7f1cb006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 -- 172.21.15.90:0/636929797 >> 172.21.15.90:0/636929797 conn(0x7f1cb00793f0 msgr2=0x7f1cb0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 -- 172.21.15.90:0/636929797 shutdown_connections 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.416+0000 7f1cb7600700 1 -- 172.21.15.90:0/636929797 wait complete. 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.417+0000 7f1cb7600700 1 Processor -- start 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 -- start start 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 0x7f1cb0199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cb006bbe0 0x7f1cb019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb01095b0 0x7f1cb019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0113790 con 0x7f1cb006b220 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0113610 con 0x7f1cb006bbe0 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb7600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0113910 con 0x7f1cb01095b0 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 0x7f1cb0199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 0x7f1cb0199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35726/0 (socket says 172.21.15.90:35726) 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb4c00700 1 -- 172.21.15.90:0/2075739371 learned_addr learned my addr 172.21.15.90:0/2075739371 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cafe00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cb006bbe0 0x7f1cb019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.418+0000 7f1cb5600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb01095b0 0x7f1cb019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 -- 172.21.15.90:0/2075739371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb01095b0 msgr2=0x7f1cb019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb01095b0 0x7f1cb019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 -- 172.21.15.90:0/2075739371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cb006bbe0 msgr2=0x7f1cb019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1cb006bbe0 0x7f1cb019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb019ed30 con 0x7f1cb006b220 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb5600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cb01095b0 0x7f1cb019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb4c00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 0x7f1cb0199d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d010 tx=0x7f1ca4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4021070 con 0x7f1cb006b220 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca4007c40 con 0x7f1cb006b220 2024-06-02T15:40:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb019efc0 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.419+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4004760 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.420+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a68a0 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.421+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1cb0063c50 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.423+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1ca4003a00 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.424+0000 7f1cad600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1c980b2990 0x7f1c980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1ca400f020 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cafe00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1c980b2990 0x7f1c980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cad600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] conn(0x7f1c980bdc60 0x7f1c980c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c980c07d0 con 0x7f1c980bdc60 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1ca40e4210 con 0x7f1cb006b220 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cb5600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] conn(0x7f1c980bdc60 0x7f1c980c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.425+0000 7f1cafe00700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1c980b2990 0x7f1c980b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1cb01084d0 tx=0x7f1c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.426+0000 7f1cb5600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] conn(0x7f1c980bdc60 0x7f1c980c0100 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.427+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== osd.3 v2:172.21.15.90:6810/3752831765 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c980c07d0 con 0x7f1c980bdc60 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.445+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 --> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1cb0064080 con 0x7f1c980bdc60 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cad600700 1 -- 172.21.15.90:0/2075739371 <== osd.3 v2:172.21.15.90:6810/3752831765 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1cb0064080 con 0x7f1c980bdc60 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] conn(0x7f1c980bdc60 msgr2=0x7f1c980c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] conn(0x7f1c980bdc60 0x7f1c980c0100 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1c980b2990 msgr2=0x7f1c980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1c980b2990 0x7f1c980b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1cb01084d0 tx=0x7f1c9c009260 comp rx=0 tx=0).stop 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 msgr2=0x7f1cb0199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.446+0000 7f1cb7600700 1 --2- 172.21.15.90:0/2075739371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cb006b220 0x7f1cb0199d20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400d010 tx=0x7f1ca4004d00 comp rx=0 tx=0).stop 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.447+0000 7f1cb5600700 1 -- 172.21.15.90:0/2075739371 reap_dead start 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.447+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 shutdown_connections 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.447+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 >> 172.21.15.90:0/2075739371 conn(0x7f1cb00793f0 msgr2=0x7f1cb0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.447+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 shutdown_connections 2024-06-02T15:40:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:50.447+0000 7f1cb7600700 1 -- 172.21.15.90:0/2075739371 wait complete. 2024-06-02T15:40:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:51 smithi120 ceph-mon[36451]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:51 smithi135 ceph-mon[36839]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:51.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:51 smithi090 ceph-mon[31595]: pgmap v406: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:53.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:53 smithi090 ceph-mon[31595]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:53 smithi120 ceph-mon[36451]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:53 smithi135 ceph-mon[36839]: pgmap v407: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 -- 172.21.15.90:0/3613078800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c109dd0 msgr2=0x7fe35c10a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 --2- 172.21.15.90:0/3613078800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c109dd0 0x7fe35c10a1b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe3500099c0 tx=0x7fe350009cd0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 -- 172.21.15.90:0/3613078800 shutdown_connections 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 --2- 172.21.15.90:0/3613078800 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe35c10eeb0 0x7fe35c116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 --2- 172.21.15.90:0/3613078800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c109dd0 0x7fe35c10a1b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 --2- 172.21.15.90:0/3613078800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.881+0000 7fe364200700 1 -- 172.21.15.90:0/3613078800 >> 172.21.15.90:0/3613078800 conn(0x7fe35c076d10 msgr2=0x7fe35c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.882+0000 7fe364200700 1 -- 172.21.15.90:0/3613078800 shutdown_connections 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.882+0000 7fe364200700 1 -- 172.21.15.90:0/3613078800 wait complete. 2024-06-02T15:40:53.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 Processor -- start 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 -- start start 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe35c109dd0 0x7fe35c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c10eeb0 0x7fe35c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c119da0 con 0x7fe35c10eeb0 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c119c20 con 0x7fe35c109dd0 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.883+0000 7fe364200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c119f20 con 0x7fe35c10a780 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe362200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c10eeb0 0x7fe35c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57462/0 (socket says 172.21.15.90:57462) 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 -- 172.21.15.90:0/1504009216 learned_addr learned my addr 172.21.15.90:0/1504009216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe361800700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe35c109dd0 0x7fe35c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:53.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 -- 172.21.15.90:0/1504009216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe35c109dd0 msgr2=0x7fe35c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe35c109dd0 0x7fe35c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 -- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c10eeb0 msgr2=0x7fe35c1a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe35c10eeb0 0x7fe35c1a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.884+0000 7fe360e00700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c1a5270 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe360e00700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c1a07f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe34800e970 tx=0x7fe34800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348004a30 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c1a5560 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe34800cc70 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348005580 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.885+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c1acde0 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.886+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe35c063c90 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.888+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe34800c7c0 con 0x7fe35c10a780 2024-06-02T15:40:53.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe35a000700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3440b2910 0x7fe3440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe3480e2dc0 con 0x7fe35c10a780 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe361800700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3440b2910 0x7fe3440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe35a000700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] conn(0x7fe3440bdbf0 0x7fe3440c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe34800be50 con 0x7fe3440bdbf0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.890+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe3480e3170 con 0x7fe35c10a780 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.891+0000 7fe362200700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] conn(0x7fe3440bdbf0 0x7fe3440c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.891+0000 7fe361800700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3440b2910 0x7fe3440b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe350003760 tx=0x7fe350005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.891+0000 7fe362200700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] conn(0x7fe3440bdbf0 0x7fe3440c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.892+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== osd.9 v2:172.21.15.90:6826/3920640313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe34800be50 con 0x7fe3440bdbf0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.910+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 --> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe35c064090 con 0x7fe3440bdbf0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe35a000700 1 -- 172.21.15.90:0/1504009216 <== osd.9 v2:172.21.15.90:6826/3920640313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe35c064090 con 0x7fe3440bdbf0 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] conn(0x7fe3440bdbf0 msgr2=0x7fe3440c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] conn(0x7fe3440bdbf0 0x7fe3440c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3440b2910 msgr2=0x7fe3440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3440b2910 0x7fe3440b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe350003760 tx=0x7fe350005fb0 comp rx=0 tx=0).stop 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 msgr2=0x7fe35c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 --2- 172.21.15.90:0/1504009216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe35c10a780 0x7fe35c1a07f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe34800e970 tx=0x7fe34800ec80 comp rx=0 tx=0).stop 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe362200700 1 -- 172.21.15.90:0/1504009216 reap_dead start 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 shutdown_connections 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.911+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 >> 172.21.15.90:0/1504009216 conn(0x7fe35c076d10 msgr2=0x7fe35c078940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.912+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 shutdown_connections 2024-06-02T15:40:53.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.912+0000 7fe364200700 1 -- 172.21.15.90:0/1504009216 wait complete. 2024-06-02T15:40:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 -- 172.21.15.90:0/2220167830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 msgr2=0x7fbe2c075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2220167830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c075b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009990 tx=0x7fbe1c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 -- 172.21.15.90:0/2220167830 shutdown_connections 2024-06-02T15:40:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2220167830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 0x7fbe2c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2220167830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c075b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2220167830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe2c076090 0x7fbe2c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 -- 172.21.15.90:0/2220167830 >> 172.21.15.90:0/2220167830 conn(0x7fbe2c07a6a0 msgr2=0x7fbe2c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 -- 172.21.15.90:0/2220167830 shutdown_connections 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.754+0000 7fbe31c00700 1 -- 172.21.15.90:0/2220167830 wait complete. 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.755+0000 7fbe31c00700 1 Processor -- start 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 -- start start 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe2c076090 0x7fbe2c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 0x7fbe2c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c115940 con 0x7fbe2c10ff40 2024-06-02T15:40:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1157c0 con 0x7fbe2c076090 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe31c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c115ac0 con 0x7fbe2c075770 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe2aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe2aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57424/0 (socket says 172.21.15.90:57424) 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe2aa00700 1 -- 172.21.15.90:0/2009450375 learned_addr learned my addr 172.21.15.90:0/2009450375 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe2a000700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe2c076090 0x7fbe2c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.756+0000 7fbe2b400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 0x7fbe2c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 -- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe2c076090 msgr2=0x7fbe2c19c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbe2c076090 0x7fbe2c19c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 -- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 msgr2=0x7fbe2c1a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 0x7fbe2c1a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c1a0e40 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2b400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe2c10ff40 0x7fbe2c1a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.757+0000 7fbe2aa00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c19be30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.758+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c027070 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.758+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1c022470 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.758+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c1a10d0 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.758+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c017650 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.758+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c1a89b0 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.759+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe2c063cd0 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.761+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbe1c02a070 con 0x7fbe2c075770 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe23400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbe100b2990 0x7fbe100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:54.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbe1c0e3670 con 0x7fbe2c075770 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe2a000700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbe100b2990 0x7fbe100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe23400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] conn(0x7fbe100bdb40 0x7fbe100bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe100c06b0 con 0x7fbe100bdb40 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.763+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c0e3a20 con 0x7fbe2c075770 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.764+0000 7fbe2a000700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbe100b2990 0x7fbe100b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c111c70 tx=0x7fbe18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.764+0000 7fbe2b400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] conn(0x7fbe100bdb40 0x7fbe100bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.764+0000 7fbe2b400700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] conn(0x7fbe100bdb40 0x7fbe100bffe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.765+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== osd.10 v2:172.21.15.120:6824/1810652587 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe100c06b0 con 0x7fbe100bdb40 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.784+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 --> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe2c064100 con 0x7fbe100bdb40 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.784+0000 7fbe23400700 1 -- 172.21.15.90:0/2009450375 <== osd.10 v2:172.21.15.120:6824/1810652587 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe2c064100 con 0x7fbe100bdb40 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] conn(0x7fbe100bdb40 msgr2=0x7fbe100bffe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] conn(0x7fbe100bdb40 0x7fbe100bffe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbe100b2990 msgr2=0x7fbe100b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbe100b2990 0x7fbe100b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c111c70 tx=0x7fbe18009260 comp rx=0 tx=0).stop 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 msgr2=0x7fbe2c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 --2- 172.21.15.90:0/2009450375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe2c075770 0x7fbe2c19be30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c000c00 tx=0x7fbe1c0057c0 comp rx=0 tx=0).stop 2024-06-02T15:40:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe2b400700 1 -- 172.21.15.90:0/2009450375 reap_dead start 2024-06-02T15:40:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 shutdown_connections 2024-06-02T15:40:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 >> 172.21.15.90:0/2009450375 conn(0x7fbe2c07a6a0 msgr2=0x7fbe2c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 shutdown_connections 2024-06-02T15:40:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:52.785+0000 7fbe31c00700 1 -- 172.21.15.90:0/2009450375 wait complete. 2024-06-02T15:40:54.933 INFO:teuthology.orchestra.run.smithi090.stdout:111669149789 2024-06-02T15:40:54.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-02T15:40:55.532 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:55 smithi090 ceph-mon[31595]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:55 smithi120 ceph-mon[36451]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:55 smithi135 ceph-mon[36839]: pgmap v408: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:40:55.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 -- 172.21.15.90:0/3210464453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 msgr2=0x7f9acc103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 --2- 172.21.15.90:0/3210464453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc103bd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0098f0 tx=0x7f9abc009c00 comp rx=0 tx=0).stop 2024-06-02T15:40:55.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 -- 172.21.15.90:0/3210464453 shutdown_connections 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 --2- 172.21.15.90:0/3210464453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc1088f0 0x7f9acc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 --2- 172.21.15.90:0/3210464453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc103bd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 --2- 172.21.15.90:0/3210464453 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9acc1041a0 0x7f9acc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.052+0000 7f9ad2200700 1 -- 172.21.15.90:0/3210464453 >> 172.21.15.90:0/3210464453 conn(0x7f9acc07a6a0 msgr2=0x7f9acc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.053+0000 7f9ad2200700 1 -- 172.21.15.90:0/3210464453 shutdown_connections 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.053+0000 7f9ad2200700 1 -- 172.21.15.90:0/3210464453 wait complete. 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 Processor -- start 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 -- start start 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9acc1041a0 0x7f9acc075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc1088f0 0x7f9acc0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:55.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc113710 con 0x7f9acc1088f0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc113590 con 0x7f9acc1041a0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.054+0000 7f9ad2200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc113890 con 0x7f9acc1037f0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc1088f0 0x7f9acc0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57436/0 (socket says 172.21.15.90:57436) 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 -- 172.21.15.90:0/1870746400 learned_addr learned my addr 172.21.15.90:0/1870746400 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acaa00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9acc1041a0 0x7f9acc075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 -- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9acc1041a0 msgr2=0x7f9acc075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9acc1041a0 0x7f9acc075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 -- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc1088f0 msgr2=0x7f9acc0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc1088f0 0x7f9acc0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.055+0000 7f9acb400700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1a8170 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9acb400700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc075840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009230 tx=0x7f9abc00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0050c0 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9abc00b560 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc003a00 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1a8400 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.056+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a8850 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.057+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9acc063cd0 con 0x7f9acc1037f0 2024-06-02T15:40:55.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.060+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9abc00b710 con 0x7f9acc1037f0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.061+0000 7f9ac3e00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ab00b2910 0x7f9ab00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.061+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9abc0e3a90 con 0x7f9acc1037f0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.061+0000 7f9acaa00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ab00b2910 0x7f9ab00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.062+0000 7f9ac3e00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] conn(0x7f9ab00bdbf0 0x7f9ab00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.062+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9abc024e50 con 0x7f9ab00bdbf0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.062+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9abc0e3e40 con 0x7f9acc1037f0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.062+0000 7f9acaa00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ab00b2910 0x7f9ab00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9acc1031b0 tx=0x7f9ab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.062+0000 7f9acbe00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] conn(0x7f9ab00bdbf0 0x7f9ab00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.063+0000 7f9acbe00700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] conn(0x7f9ab00bdbf0 0x7f9ab00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.064+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== osd.1 v2:172.21.15.120:6800/3632063452 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9abc024e50 con 0x7f9ab00bdbf0 2024-06-02T15:40:55.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.081+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 --> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9acc064150 con 0x7f9ab00bdbf0 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.082+0000 7f9ac3e00700 1 -- 172.21.15.90:0/1870746400 <== osd.1 v2:172.21.15.120:6800/3632063452 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9acc064150 con 0x7f9ab00bdbf0 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] conn(0x7f9ab00bdbf0 msgr2=0x7f9ab00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] conn(0x7f9ab00bdbf0 0x7f9ab00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ab00b2910 msgr2=0x7f9ab00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ab00b2910 0x7f9ab00b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9acc1031b0 tx=0x7f9ab8009260 comp rx=0 tx=0).stop 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 msgr2=0x7f9acc075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 --2- 172.21.15.90:0/1870746400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9acc1037f0 0x7f9acc075840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009230 tx=0x7f9abc00fa70 comp rx=0 tx=0).stop 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9acbe00700 1 -- 172.21.15.90:0/1870746400 reap_dead start 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 shutdown_connections 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 >> 172.21.15.90:0/1870746400 conn(0x7f9acc07a6a0 msgr2=0x7f9acc1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 shutdown_connections 2024-06-02T15:40:55.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:53.083+0000 7f9ad2200700 1 -- 172.21.15.90:0/1870746400 wait complete. 2024-06-02T15:40:55.938 INFO:teuthology.orchestra.run.smithi090.stdout:231928234050 2024-06-02T15:40:55.938 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-02T15:40:57.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:57 smithi090 ceph-mon[31595]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:57 smithi120 ceph-mon[36451]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:57 smithi135 ceph-mon[36839]: pgmap v409: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:40:57.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 -- 172.21.15.90:0/2140139047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 msgr2=0x7f230c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 --2- 172.21.15.90:0/2140139047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c109920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0099c0 tx=0x7f22fc009cd0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 -- 172.21.15.90:0/2140139047 shutdown_connections 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 --2- 172.21.15.90:0/2140139047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103e50 0x7f230c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 --2- 172.21.15.90:0/2140139047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c109920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 --2- 172.21.15.90:0/2140139047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f230c103490 0x7f230c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 -- 172.21.15.90:0/2140139047 >> 172.21.15.90:0/2140139047 conn(0x7f230c0fefc0 msgr2=0x7f230c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 -- 172.21.15.90:0/2140139047 shutdown_connections 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.741+0000 7f2311400700 1 -- 172.21.15.90:0/2140139047 wait complete. 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.742+0000 7f2311400700 1 Processor -- start 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 -- start start 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103490 0x7f230c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f230c103e50 0x7f230c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:57.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f230c113670 con 0x7f230c103e50 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1134f0 con 0x7f230c109540 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2311400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1137f0 con 0x7f230c103490 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f230b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f230c103e50 0x7f230c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2310a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f230be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103490 0x7f230c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2310a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:45820/0 (socket says 172.21.15.90:45820) 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f2310a00700 1 -- 172.21.15.90:0/3472426144 learned_addr learned my addr 172.21.15.90:0/3472426144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.743+0000 7f230be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103490 0x7f230c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57478/0 (socket says 172.21.15.90:57478) 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 -- 172.21.15.90:0/3472426144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103490 msgr2=0x7f230c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c103490 0x7f230c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 -- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f230c103e50 msgr2=0x7f230c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f230c103e50 0x7f230c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c1a0d40 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2310a00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c1a0600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f230400ef30 tx=0x7f230400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040092e0 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.745+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2304004bb0 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.744+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c1a0fd0 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.745+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304004020 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.745+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1a88b0 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.745+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f230c100e40 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.748+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f230401b030 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f2308c00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f22f00b2910 0x7f22f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f23040e2be0 con 0x7f230c109540 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f230be00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f22f00b2910 0x7f22f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f2308c00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] conn(0x7f22f00bdbf0 0x7f22f00c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:57.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22f00c0760 con 0x7f22f00bdbf0 2024-06-02T15:40:57.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f23040e2f90 con 0x7f230c109540 2024-06-02T15:40:57.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.750+0000 7f230b400700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] conn(0x7f22f00bdbf0 0x7f22f00c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:57.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.751+0000 7f230be00700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f22f00b2910 0x7f22f00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22fc000c00 tx=0x7f22fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:57.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.751+0000 7f230b400700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] conn(0x7f22f00bdbf0 0x7f22f00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:57.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.752+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== osd.6 v2:172.21.15.90:6818/3483502077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22f00c0760 con 0x7f22f00bdbf0 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.770+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 --> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f230c104bb0 con 0x7f22f00bdbf0 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.771+0000 7f2308c00700 1 -- 172.21.15.90:0/3472426144 <== osd.6 v2:172.21.15.90:6818/3483502077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f230c104bb0 con 0x7f22f00bdbf0 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.771+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] conn(0x7f22f00bdbf0 msgr2=0x7f22f00c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.771+0000 7f2311400700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] conn(0x7f22f00bdbf0 0x7f22f00c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f22f00b2910 msgr2=0x7f22f00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f22f00b2910 0x7f22f00b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22fc000c00 tx=0x7f22fc005fb0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 msgr2=0x7f230c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 --2- 172.21.15.90:0/3472426144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f230c109540 0x7f230c1a0600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f230400ef30 tx=0x7f230400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2310a00700 1 -- 172.21.15.90:0/3472426144 reap_dead start 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 shutdown_connections 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 >> 172.21.15.90:0/3472426144 conn(0x7f230c0fefc0 msgr2=0x7f230c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 shutdown_connections 2024-06-02T15:40:57.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:56.772+0000 7f2311400700 1 -- 172.21.15.90:0/3472426144 wait complete. 2024-06-02T15:40:58.861 INFO:teuthology.orchestra.run.smithi090.stdout:206158430279 2024-06-02T15:40:58.862 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 -- 172.21.15.90:0/1130338170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 msgr2=0x7fe8bc06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1130338170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc06b630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009990 tx=0x7fe8ac009ca0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 -- 172.21.15.90:0/1130338170 shutdown_connections 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1130338170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8bc10fff0 0x7fe8bc1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1130338170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc06b630 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1130338170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8bc06bb70 0x7fe8bc10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.000+0000 7fe8c2800700 1 -- 172.21.15.90:0/1130338170 >> 172.21.15.90:0/1130338170 conn(0x7fe8bc0793f0 msgr2=0x7fe8bc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.001+0000 7fe8c2800700 1 -- 172.21.15.90:0/1130338170 shutdown_connections 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.001+0000 7fe8c2800700 1 -- 172.21.15.90:0/1130338170 wait complete. 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 Processor -- start 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 -- start start 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8bc06bb70 0x7fe8bc19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8bc10fff0 0x7fe8bc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115910 con 0x7fe8bc06b250 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115790 con 0x7fe8bc06bb70 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.002+0000 7fe8c2800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc115a90 con 0x7fe8bc10fff0 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc19be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40568/0 (socket says 172.21.15.90:40568) 2024-06-02T15:40:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 -- 172.21.15.90:0/1237253427 learned_addr learned my addr 172.21.15.90:0/1237253427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bbe00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8bc10fff0 0x7fe8bc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8baa00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8bc06bb70 0x7fe8bc19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 -- 172.21.15.90:0/1237253427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8bc10fff0 msgr2=0x7fe8bc1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8bc10fff0 0x7fe8bc1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 -- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8bc06bb70 msgr2=0x7fe8bc19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8bc06bb70 0x7fe8bc19c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.003+0000 7fe8bb400700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc1a0e30 con 0x7fe8bc06b250 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8bb400700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc19be20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0057f0 tx=0x7fe8ac00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac026070 con 0x7fe8bc06b250 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc1a10c0 con 0x7fe8bc06b250 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac021410 con 0x7fe8bc06b250 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac017650 con 0x7fe8bc06b250 2024-06-02T15:40:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.004+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc1a89a0 con 0x7fe8bc06b250 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.005+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8bc063c50 con 0x7fe8bc06b250 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.007+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe8ac029070 con 0x7fe8bc06b250 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.009+0000 7fe8b3e00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8a00b2990 0x7fe8a00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.009+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe8ac0e3760 con 0x7fe8bc06b250 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.009+0000 7fe8baa00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8a00b2990 0x7fe8a00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.010+0000 7fe8b3e00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] conn(0x7fe8a00bdc70 0x7fe8a00c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.010+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8ac001d30 con 0x7fe8a00bdc70 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.010+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac0e3b10 con 0x7fe8bc06b250 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.010+0000 7fe8baa00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8a00b2990 0x7fe8a00b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc1039a0 tx=0x7fe8a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.010+0000 7fe8bbe00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] conn(0x7fe8a00bdc70 0x7fe8a00c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.011+0000 7fe8bbe00700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] conn(0x7fe8a00bdc70 0x7fe8a00c0110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.012+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== osd.4 v2:172.21.15.120:6808/3009183394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8ac001d30 con 0x7fe8a00bdc70 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.029+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 --> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8bc064050 con 0x7fe8a00bdc70 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.030+0000 7fe8b3e00700 1 -- 172.21.15.90:0/1237253427 <== osd.4 v2:172.21.15.120:6808/3009183394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe8bc064050 con 0x7fe8a00bdc70 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] conn(0x7fe8a00bdc70 msgr2=0x7fe8a00c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] conn(0x7fe8a00bdc70 0x7fe8a00c0110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8a00b2990 msgr2=0x7fe8a00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8a00b2990 0x7fe8a00b4e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc1039a0 tx=0x7fe8a8009260 comp rx=0 tx=0).stop 2024-06-02T15:40:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 msgr2=0x7fe8bc19be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 --2- 172.21.15.90:0/1237253427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8bc06b250 0x7fe8bc19be20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0057f0 tx=0x7fe8ac00bfd0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8bbe00700 1 -- 172.21.15.90:0/1237253427 reap_dead start 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 shutdown_connections 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 >> 172.21.15.90:0/1237253427 conn(0x7fe8bc0793f0 msgr2=0x7fe8bc1114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 shutdown_connections 2024-06-02T15:40:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:58.031+0000 7fe8c2800700 1 -- 172.21.15.90:0/1237253427 wait complete. 2024-06-02T15:40:59.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:40:59 smithi090 ceph-mon[31595]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:40:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:40:59 smithi120 ceph-mon[36451]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:40:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:40:59 smithi135 ceph-mon[36839]: pgmap v410: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:40:59.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 -- 172.21.15.90:0/3764051369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281041a0 msgr2=0x7f96281081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 --2- 172.21.15.90:0/3764051369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281041a0 0x7f96281081c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9614009920 tx=0x7f9614009c30 comp rx=0 tx=0).stop 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 -- 172.21.15.90:0/3764051369 shutdown_connections 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 --2- 172.21.15.90:0/3764051369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96281088f0 0x7f9628110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 --2- 172.21.15.90:0/3764051369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96281037f0 0x7f9628103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 --2- 172.21.15.90:0/3764051369 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281041a0 0x7f96281081c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 -- 172.21.15.90:0/3764051369 >> 172.21.15.90:0/3764051369 conn(0x7f962807a6a0 msgr2=0x7f962807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 -- 172.21.15.90:0/3764051369 shutdown_connections 2024-06-02T15:40:59.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.745+0000 7f962e600700 1 -- 172.21.15.90:0/3764051369 wait complete. 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.746+0000 7f962e600700 1 Processor -- start 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.746+0000 7f962e600700 1 -- start start 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 0x7f962819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96281041a0 0x7f962819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96281088f0 0x7f96281a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9628113720 con 0x7f96281088f0 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96281135a0 con 0x7f96281037f0 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f962e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96281138a0 con 0x7f96281041a0 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f9627400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 0x7f962819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f9627400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 0x7f962819be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:45852/0 (socket says 172.21.15.90:45852) 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f9627400700 1 -- 172.21.15.90:0/734902726 learned_addr learned my addr 172.21.15.90:0/734902726 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:40:59.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f9627e00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96281088f0 0x7f96281a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.747+0000 7f9626a00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96281041a0 0x7f962819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 -- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96281041a0 msgr2=0x7f962819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96281041a0 0x7f962819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 -- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96281088f0 msgr2=0x7f96281a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96281088f0 0x7f96281a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96281a0e80 con 0x7f96281037f0 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f9627400700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 0x7f962819be70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f961800ec10 tx=0x7f961800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961800cc90 con 0x7f96281037f0 2024-06-02T15:40:59.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f961800ce30 con 0x7f96281037f0 2024-06-02T15:40:59.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618010430 con 0x7f96281037f0 2024-06-02T15:40:59.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96281a1170 con 0x7f96281037f0 2024-06-02T15:40:59.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.748+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a89f0 con 0x7f96281037f0 2024-06-02T15:40:59.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.749+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9628076ad0 con 0x7f96281037f0 2024-06-02T15:40:59.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.752+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9618004000 con 0x7f96281037f0 2024-06-02T15:40:59.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f961fe00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f960c0b2990 0x7f960c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f96180e2df0 con 0x7f96281037f0 2024-06-02T15:40:59.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f9626a00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f960c0b2990 0x7f960c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f961fe00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] conn(0x7f960c0bdc70 0x7f960c0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:40:59.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9618015e80 con 0x7f960c0bdc70 2024-06-02T15:40:59.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.754+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f96180e31a0 con 0x7f96281037f0 2024-06-02T15:40:59.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.755+0000 7f9627e00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] conn(0x7f960c0bdc70 0x7f960c0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:40:59.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.755+0000 7f9626a00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f960c0b2990 0x7f960c0b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9628101440 tx=0x7f9614019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.756+0000 7f9627e00700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] conn(0x7f960c0bdc70 0x7f960c0c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:40:59.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.757+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== osd.11 v2:172.21.15.135:6824/359143967 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9618015e80 con 0x7f960c0bdc70 2024-06-02T15:40:59.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.786+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 --> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9628104c80 con 0x7f960c0bdc70 2024-06-02T15:40:59.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.787+0000 7f961fe00700 1 -- 172.21.15.90:0/734902726 <== osd.11 v2:172.21.15.135:6824/359143967 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9628104c80 con 0x7f960c0bdc70 2024-06-02T15:40:59.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.787+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] conn(0x7f960c0bdc70 msgr2=0x7f960c0c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.787+0000 7f962e600700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] conn(0x7f960c0bdc70 0x7f960c0c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f960c0b2990 msgr2=0x7f960c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f960c0b2990 0x7f960c0b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9628101440 tx=0x7f9614019040 comp rx=0 tx=0).stop 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 msgr2=0x7f962819be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 --2- 172.21.15.90:0/734902726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96281037f0 0x7f962819be70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f961800ec10 tx=0x7f961800ef20 comp rx=0 tx=0).stop 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f9627e00700 1 -- 172.21.15.90:0/734902726 reap_dead start 2024-06-02T15:40:59.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 shutdown_connections 2024-06-02T15:40:59.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 >> 172.21.15.90:0/734902726 conn(0x7f962807a6a0 msgr2=0x7f9628101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:40:59.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 shutdown_connections 2024-06-02T15:40:59.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.788+0000 7f962e600700 1 -- 172.21.15.90:0/734902726 wait complete. 2024-06-02T15:41:00.529 INFO:teuthology.orchestra.run.smithi090.stdout:279172874294 2024-06-02T15:41:00.530 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-02T15:41:01.171 INFO:teuthology.orchestra.run.smithi090.stdout:184683593806 2024-06-02T15:41:01.172 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-02T15:41:01.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:01 smithi090 ceph-mon[31595]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:01 smithi120 ceph-mon[36451]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:01 smithi135 ceph-mon[36839]: pgmap v411: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:01.987 INFO:teuthology.orchestra.run.smithi090.stdout:257698037820 2024-06-02T15:41:01.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-02T15:41:02.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.789+0000 7f7924600700 1 -- 172.21.15.90:0/2496824328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c10a780 msgr2=0x7f791c10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.789+0000 7f7924600700 1 --2- 172.21.15.90:0/2496824328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c10a780 0x7f791c10e7d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f79080099c0 tx=0x7f7908009cd0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 -- 172.21.15.90:0/2496824328 shutdown_connections 2024-06-02T15:41:02.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 --2- 172.21.15.90:0/2496824328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f791c10eeb0 0x7f791c116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 --2- 172.21.15.90:0/2496824328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791c109dd0 0x7f791c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 --2- 172.21.15.90:0/2496824328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c10a780 0x7f791c10e7d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 -- 172.21.15.90:0/2496824328 >> 172.21.15.90:0/2496824328 conn(0x7f791c076d10 msgr2=0x7f791c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 -- 172.21.15.90:0/2496824328 shutdown_connections 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.790+0000 7f7924600700 1 -- 172.21.15.90:0/2496824328 wait complete. 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 Processor -- start 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 -- start start 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 0x7f791c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f791c10a780 0x7f791c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791c10eeb0 0x7f791c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f791c119d50 con 0x7f791c109dd0 2024-06-02T15:41:02.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f791c119bd0 con 0x7f791c10a780 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7924600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f791c119ed0 con 0x7f791c10eeb0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7921c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 0x7f791c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7921c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 0x7f791c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47374/0 (socket says 172.21.15.90:47374) 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.791+0000 7f7921c00700 1 -- 172.21.15.90:0/1194046771 learned_addr learned my addr 172.21.15.90:0/1194046771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7922600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791c10eeb0 0x7f791c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921200700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f791c10a780 0x7f791c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 -- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791c10eeb0 msgr2=0x7f791c1a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f791c10eeb0 0x7f791c1a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 -- 172.21.15.90:0/1194046771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f791c10a780 msgr2=0x7f791c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f791c10a780 0x7f791c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c1a5290 con 0x7f791c109dd0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f7921c00700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 0x7f791c1a0280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f791000e9c0 tx=0x7f791000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:02.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.792+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791000cbd0 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.793+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c1a5580 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.793+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791000ee60 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.793+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100055c0 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.793+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1ace00 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.794+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f791c063c50 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.796+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7910004750 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.797+0000 7f791a000700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f79040b2910 0x7f79040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f79100e2e40 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f7921200700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f79040b2910 0x7f79040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f791a000700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] conn(0x7f79040bdbf0 0x7f79040c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f791000be50 con 0x7f79040bdbf0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f79100e31f0 con 0x7f791c109dd0 2024-06-02T15:41:02.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f7922600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] conn(0x7f79040bdbf0 0x7f79040c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.798+0000 7f7921200700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f79040b2910 0x7f79040b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7908000c00 tx=0x7f7908005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.799+0000 7f7922600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] conn(0x7f79040bdbf0 0x7f79040c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.800+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== osd.5 v2:172.21.15.135:6808/3338529277 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f791000be50 con 0x7f79040bdbf0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.818+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 --> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f791c0640d0 con 0x7f79040bdbf0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.818+0000 7f791a000700 1 -- 172.21.15.90:0/1194046771 <== osd.5 v2:172.21.15.135:6808/3338529277 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f791c0640d0 con 0x7f79040bdbf0 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] conn(0x7f79040bdbf0 msgr2=0x7f79040c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] conn(0x7f79040bdbf0 0x7f79040c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f79040b2910 msgr2=0x7f79040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f79040b2910 0x7f79040b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7908000c00 tx=0x7f7908005fb0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 msgr2=0x7f791c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 --2- 172.21.15.90:0/1194046771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f791c109dd0 0x7f791c1a0280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f791000e9c0 tx=0x7f791000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7922600700 1 -- 172.21.15.90:0/1194046771 reap_dead start 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 shutdown_connections 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 >> 172.21.15.90:0/1194046771 conn(0x7f791c076d10 msgr2=0x7f791c0773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 shutdown_connections 2024-06-02T15:41:02.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:02.819+0000 7f7924600700 1 -- 172.21.15.90:0/1194046771 wait complete. 2024-06-02T15:41:03.213 INFO:teuthology.orchestra.run.smithi090.stdout:55834574954 2024-06-02T15:41:03.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 -- 172.21.15.90:0/2338455777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f536810b780 msgr2=0x7f536810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 --2- 172.21.15.90:0/2338455777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f536810b780 0x7f536810bb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53580098f0 tx=0x7f5358009c00 comp rx=0 tx=0).stop 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 -- 172.21.15.90:0/2338455777 shutdown_connections 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 --2- 172.21.15.90:0/2338455777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f5368112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 --2- 172.21.15.90:0/2338455777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f536810b780 0x7f536810bb60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 --2- 172.21.15.90:0/2338455777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5368105780 0x7f5368105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 -- 172.21.15.90:0/2338455777 >> 172.21.15.90:0/2338455777 conn(0x7f5368101270 msgr2=0x7f5368103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 -- 172.21.15.90:0/2338455777 shutdown_connections 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.993+0000 7f536ee00700 1 -- 172.21.15.90:0/2338455777 wait complete. 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.994+0000 7f536ee00700 1 Processor -- start 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.994+0000 7f536ee00700 1 -- start start 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5368105780 0x7f536819bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f536819c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 0x7f53681a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115980 con 0x7f5368106140 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115800 con 0x7f536810b780 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ee00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115b00 con 0x7f5368105780 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f535fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f536819c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:03.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ce00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 0x7f53681a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f5367e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5368105780 0x7f536819bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f535fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f536819c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40602/0 (socket says 172.21.15.90:40602) 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f536ce00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 0x7f53681a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:45882/0 (socket says 172.21.15.90:45882) 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.995+0000 7f535fe00700 1 -- 172.21.15.90:0/844920970 learned_addr learned my addr 172.21.15.90:0/844920970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 -- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5368105780 msgr2=0x7f536819bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5368105780 0x7f536819bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 -- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 msgr2=0x7f536819c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f536819c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681a0e90 con 0x7f536810b780 2024-06-02T15:41:03.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f535fe00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5368106140 0x7f536819c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ce00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 0x7f53681a0750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f536000efc0 tx=0x7f536000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600092e0 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681a1120 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5360004bb0 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360004020 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.996+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a8a00 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:40:59.997+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5368063cd0 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.000+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f536001b030 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.001+0000 7f5365600700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f534c0b2990 0x7f534c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.001+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f53600e2aa0 con 0x7f536810b780 2024-06-02T15:41:03.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.001+0000 7f5367e00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f534c0b2990 0x7f534c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.002+0000 7f5365600700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] conn(0x7f534c0bdc70 0x7f534c0c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.002+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5360011e50 con 0x7f534c0bdc70 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.002+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f53600e2e50 con 0x7f536810b780 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.002+0000 7f535fe00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] conn(0x7f534c0bdc70 0x7f534c0c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.002+0000 7f5367e00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f534c0b2990 0x7f534c0b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5358009fd0 tx=0x7f5358019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.003+0000 7f535fe00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] conn(0x7f534c0bdc70 0x7f534c0c0110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.003+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== osd.2 v2:172.21.15.135:6800/3984900996 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5360011e50 con 0x7f534c0bdc70 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.022+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 --> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5368064150 con 0x7f534c0bdc70 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.023+0000 7f5365600700 1 -- 172.21.15.90:0/844920970 <== osd.2 v2:172.21.15.135:6800/3984900996 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5368064150 con 0x7f534c0bdc70 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] conn(0x7f534c0bdc70 msgr2=0x7f534c0c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] conn(0x7f534c0bdc70 0x7f534c0c0110 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f534c0b2990 msgr2=0x7f534c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f534c0b2990 0x7f534c0b4e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5358009fd0 tx=0x7f5358019040 comp rx=0 tx=0).stop 2024-06-02T15:41:03.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 msgr2=0x7f53681a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 --2- 172.21.15.90:0/844920970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f536810b780 0x7f53681a0750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f536000efc0 tx=0x7f536000c3a0 comp rx=0 tx=0).stop 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ce00700 1 -- 172.21.15.90:0/844920970 reap_dead start 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 shutdown_connections 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 >> 172.21.15.90:0/844920970 conn(0x7f5368101270 msgr2=0x7f5368101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 shutdown_connections 2024-06-02T15:41:03.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:00.024+0000 7f536ee00700 1 -- 172.21.15.90:0/844920970 wait complete. 2024-06-02T15:41:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:03 smithi120 ceph-mon[36451]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:03 smithi135 ceph-mon[36839]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:03 smithi090 ceph-mon[31595]: pgmap v412: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:04.116 INFO:teuthology.orchestra.run.smithi090.stdout:77309411431 2024-06-02T15:41:04.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-02T15:41:04.909 INFO:teuthology.orchestra.run.smithi090.stdout:137438953562 2024-06-02T15:41:04.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-02T15:41:05.275 INFO:teuthology.orchestra.run.smithi090.stdout:304942678068 2024-06-02T15:41:05.275 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-02T15:41:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:05 smithi120 ceph-mon[36451]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:05 smithi135 ceph-mon[36839]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:05 smithi090 ceph-mon[31595]: pgmap v413: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:05.832 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:05.891 INFO:teuthology.orchestra.run.smithi090.stdout:158913790037 2024-06-02T15:41:05.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-02T15:41:06.327 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:07 smithi120 ceph-mon[36451]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:07 smithi135 ceph-mon[36839]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:07.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:07 smithi090 ceph-mon[31595]: pgmap v414: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:07.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.603+0000 7fab2ee00700 1 -- 172.21.15.90:0/1002287500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab28109a80 msgr2=0x7fab28109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.603+0000 7fab2ee00700 1 --2- 172.21.15.90:0/1002287500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab28109a80 0x7fab28109f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fab14009990 tx=0x7fab14009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 -- 172.21.15.90:0/1002287500 shutdown_connections 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 --2- 172.21.15.90:0/1002287500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab2810a440 0x7fab281168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 --2- 172.21.15.90:0/1002287500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab2810e950 0x7fab2810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 --2- 172.21.15.90:0/1002287500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab28109a80 0x7fab28109f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 -- 172.21.15.90:0/1002287500 >> 172.21.15.90:0/1002287500 conn(0x7fab280792d0 msgr2=0x7fab28108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 -- 172.21.15.90:0/1002287500 shutdown_connections 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.604+0000 7fab2ee00700 1 -- 172.21.15.90:0/1002287500 wait complete. 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.605+0000 7fab2ee00700 1 Processor -- start 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.605+0000 7fab2ee00700 1 -- start start 2024-06-02T15:41:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.605+0000 7fab2ee00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 0x7fab281a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.605+0000 7fab2ee00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab2810a440 0x7fab281a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.605+0000 7fab2ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab2810e950 0x7fab281a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab28119c80 con 0x7fab2810e950 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2ee00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fab28119b00 con 0x7fab28109a80 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2ee00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab28119e00 con 0x7fab2810a440 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab2810e950 0x7fab281a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 0x7fab281a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab2810a440 0x7fab281a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 0x7fab281a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39988/0 (socket says 172.21.15.90:39988) 2024-06-02T15:41:07.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 -- 172.21.15.90:0/124330335 learned_addr learned my addr 172.21.15.90:0/124330335 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 -- 172.21.15.90:0/124330335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab2810a440 msgr2=0x7fab281a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab2810a440 0x7fab281a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 -- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab2810e950 msgr2=0x7fab281a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab2810e950 0x7fab281a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.606+0000 7fab2da00700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab281a5160 con 0x7fab28109a80 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab2da00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 0x7fab281a0150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c016070 con 0x7fab28109a80 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab281a5450 con 0x7fab28109a80 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c004830 con 0x7fab28109a80 2024-06-02T15:41:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005540 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.607+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab281accd0 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.608+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab28063cd0 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.611+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fab1c004220 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.612+0000 7fab26000700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fab100b2990 0x7fab100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fab1c0e2b20 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab2d000700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fab100b2990 0x7fab100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab26000700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] conn(0x7fab100bdc70 0x7fab100c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab1c015ec0 con 0x7fab100bdc70 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fab1c0e2ed0 con 0x7fab28109a80 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab2e400700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] conn(0x7fab100bdc70 0x7fab100c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.613+0000 7fab2d000700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fab100b2990 0x7fab100b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab28109580 tx=0x7fab1400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.614+0000 7fab2e400700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] conn(0x7fab100bdc70 0x7fab100c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.615+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== osd.0 v2:172.21.15.90:6802/445134562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab1c015ec0 con 0x7fab100bdc70 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.633+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 --> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab28064100 con 0x7fab100bdc70 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.633+0000 7fab26000700 1 -- 172.21.15.90:0/124330335 <== osd.0 v2:172.21.15.90:6802/445134562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fab28064100 con 0x7fab100bdc70 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] conn(0x7fab100bdc70 msgr2=0x7fab100c0110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] conn(0x7fab100bdc70 0x7fab100c0110 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:07.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fab100b2990 msgr2=0x7fab100b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fab100b2990 0x7fab100b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab28109580 tx=0x7fab1400b260 comp rx=0 tx=0).stop 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 msgr2=0x7fab281a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 --2- 172.21.15.90:0/124330335 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fab28109a80 0x7fab281a0150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).stop 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2e400700 1 -- 172.21.15.90:0/124330335 reap_dead start 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 shutdown_connections 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 >> 172.21.15.90:0/124330335 conn(0x7fab280792d0 msgr2=0x7fab280798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 shutdown_connections 2024-06-02T15:41:07.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:06.634+0000 7fab2ee00700 1 -- 172.21.15.90:0/124330335 wait complete. 2024-06-02T15:41:08.005 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:08.006 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:08.006 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:08.010 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:09.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:09 smithi090 ceph-mon[31595]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:09 smithi120 ceph-mon[36451]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:09 smithi135 ceph-mon[36839]: pgmap v415: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.553+0000 7f2cd5400700 1 -- 172.21.15.90:0/1280830056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd006b250 msgr2=0x7f2cd006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.553+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1280830056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd006b250 0x7f2cd006b630 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009990 tx=0x7f2cc0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 -- 172.21.15.90:0/1280830056 shutdown_connections 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1280830056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd010fff0 0x7f2cd01123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1280830056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd006b250 0x7f2cd006b630 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1280830056 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 0x7f2cd010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 -- 172.21.15.90:0/1280830056 >> 172.21.15.90:0/1280830056 conn(0x7f2cd00793f0 msgr2=0x7f2cd0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 -- 172.21.15.90:0/1280830056 shutdown_connections 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.554+0000 7f2cd5400700 1 -- 172.21.15.90:0/1280830056 wait complete. 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.555+0000 7f2cd5400700 1 Processor -- start 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.555+0000 7f2cd5400700 1 -- start start 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.555+0000 7f2cd5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.555+0000 7f2cd5400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 0x7f2cd019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cd5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cd5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0115910 con 0x7f2cd010fff0 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cd5400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0115790 con 0x7f2cd006bb70 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cd5400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd0115a90 con 0x7f2cd006b250 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2ccf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2ccf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47426/0 (socket says 172.21.15.90:47426) 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2ccf200700 1 -- 172.21.15.90:0/1304232359 learned_addr learned my addr 172.21.15.90:0/1304232359 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:36314/0 (socket says 172.21.15.90:36314) 2024-06-02T15:41:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2ccde00700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 0x7f2cd019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 -- 172.21.15.90:0/1304232359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 msgr2=0x7f2cd019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 0x7f2cd019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 -- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 msgr2=0x7f2cd01a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2cce800700 1 -- 172.21.15.90:0/1304232359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd01a0e30 con 0x7f2cd006b250 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.556+0000 7f2ccf200700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cce800700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006010 tx=0x7f2cc00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0027070 con 0x7f2cd006b250 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd01a10c0 con 0x7f2cd006b250 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cc0022470 con 0x7f2cd006b250 2024-06-02T15:41:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0017650 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.557+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd01a89a0 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.558+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd006a230 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.561+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2cc002a070 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.562+0000 7f2cc7400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 0x7f2cb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.562+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2cc00e3400 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.562+0000 7f2ccde00700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 0x7f2cb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.563+0000 7f2ccde00700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 0x7f2cb40b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2cd01039a0 tx=0x7f2cbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.564+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cc0013070 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.804+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2cd002e560 con 0x7f2cd006b250 2024-06-02T15:41:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.804+0000 7f2cc7400700 1 -- 172.21.15.90:0/1304232359 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2cc00aac40 con 0x7f2cd006b250 2024-06-02T15:41:10.816 INFO:teuthology.orchestra.run.smithi090.stdout:111669149793 2024-06-02T15:41:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 msgr2=0x7f2cb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 0x7f2cb40b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2cd01039a0 tx=0x7f2cbc009260 comp rx=0 tx=0).stop 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 msgr2=0x7f2cd019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006010 tx=0x7f2cc00057c0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 shutdown_connections 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2cb40b2910 0x7f2cb40b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd010fff0 0x7f2cd01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cd006b250 0x7f2cd019be20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 --2- 172.21.15.90:0/1304232359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2cd006bb70 0x7f2cd019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:10.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 >> 172.21.15.90:0/1304232359 conn(0x7f2cd00793f0 msgr2=0x7f2cd01114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:10.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 shutdown_connections 2024-06-02T15:41:10.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:10.808+0000 7f2cd5400700 1 -- 172.21.15.90:0/1304232359 wait complete. 2024-06-02T15:41:11.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:11 smithi090 ceph-mon[31595]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-06-02T15:41:11.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1304232359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T15:41:11.535 INFO:teuthology.orchestra.run.smithi090.stdout:34359738481 2024-06-02T15:41:11.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-02T15:41:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:11 smithi120 ceph-mon[36451]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-06-02T15:41:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1304232359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T15:41:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:11 smithi135 ceph-mon[36839]: pgmap v416: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s 2024-06-02T15:41:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1304232359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T15:41:11.596 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:12.249 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:12.251 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:13 smithi090 ceph-mon[31595]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:13 smithi120 ceph-mon[36451]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:13 smithi135 ceph-mon[36839]: pgmap v417: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.433+0000 7f1e2be00700 1 -- 172.21.15.90:0/4074634858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c076290 msgr2=0x7f1e2c10f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.433+0000 7f1e2be00700 1 --2- 172.21.15.90:0/4074634858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c076290 0x7f1e2c10f770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009990 tx=0x7f1e14009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.434+0000 7f1e2be00700 1 -- 172.21.15.90:0/4074634858 shutdown_connections 2024-06-02T15:41:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.434+0000 7f1e2be00700 1 --2- 172.21.15.90:0/4074634858 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c10fcb0 0x7f1e2c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.434+0000 7f1e2be00700 1 --2- 172.21.15.90:0/4074634858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c075970 0x7f1e2c075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.434+0000 7f1e2be00700 1 --2- 172.21.15.90:0/4074634858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c076290 0x7f1e2c10f770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.434+0000 7f1e2be00700 1 -- 172.21.15.90:0/4074634858 >> 172.21.15.90:0/4074634858 conn(0x7f1e2c0fefc0 msgr2=0x7f1e2c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:15.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:15 smithi090 ceph-mon[31595]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.435+0000 7f1e2be00700 1 -- 172.21.15.90:0/4074634858 shutdown_connections 2024-06-02T15:41:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.435+0000 7f1e2be00700 1 -- 172.21.15.90:0/4074634858 wait complete. 2024-06-02T15:41:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.436+0000 7f1e2be00700 1 Processor -- start 2024-06-02T15:41:15.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.436+0000 7f1e2be00700 1 -- start start 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.436+0000 7f1e2be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 0x7f1e2c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c10fcb0 0x7f1e2c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c115880 con 0x7f1e2c075970 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2be00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c115700 con 0x7f1e2c076290 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2be00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c115a00 con 0x7f1e2c10fcb0 2024-06-02T15:41:15.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e23e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 0x7f1e2c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42690/0 (socket says 172.21.15.90:42690) 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2aa00700 1 -- 172.21.15.90:0/2469928842 learned_addr learned my addr 172.21.15.90:0/2469928842 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.437+0000 7f1e2b400700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c10fcb0 0x7f1e2c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 -- 172.21.15.90:0/2469928842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c10fcb0 msgr2=0x7f1e2c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c10fcb0 0x7f1e2c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 -- 172.21.15.90:0/2469928842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 msgr2=0x7f1e2c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 0x7f1e2c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 -- 172.21.15.90:0/2469928842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c1a0d60 con 0x7f1e2c075970 2024-06-02T15:41:15.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e23e00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 0x7f1e2c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2aa00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00eb30 tx=0x7f1e1c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c00cbd0 con 0x7f1e2c075970 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.438+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c1a1050 con 0x7f1e2c075970 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.439+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c1a88d0 con 0x7f1e2c075970 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.439+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1c0044d0 con 0x7f1e2c075970 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.439+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003e30 con 0x7f1e2c075970 2024-06-02T15:41:15.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.440+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e08005330 con 0x7f1e2c075970 2024-06-02T15:41:15.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.442+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1e1c004000 con 0x7f1e2c075970 2024-06-02T15:41:15.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.444+0000 7f1e30a00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 0x7f1e100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:15.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.444+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1e1c0e2a60 con 0x7f1e2c075970 2024-06-02T15:41:15.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.444+0000 7f1e23e00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 0x7f1e100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:15.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.445+0000 7f1e23e00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 0x7f1e100b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c111150 tx=0x7f1e1400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:15.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.449+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e1c0042f0 con 0x7f1e2c075970 2024-06-02T15:41:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:15 smithi120 ceph-mon[36451]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:15 smithi135 ceph-mon[36839]: pgmap v418: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:15.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.707+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1e080058f0 con 0x7f1e2c075970 2024-06-02T15:41:15.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.708+0000 7f1e30a00700 1 -- 172.21.15.90:0/2469928842 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1e1c0aa210 con 0x7f1e2c075970 2024-06-02T15:41:15.706 INFO:teuthology.orchestra.run.smithi090.stdout:77309411434 2024-06-02T15:41:15.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 msgr2=0x7f1e100b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:15.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 0x7f1e100b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c111150 tx=0x7f1e1400b260 comp rx=0 tx=0).stop 2024-06-02T15:41:15.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 msgr2=0x7f1e2c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:15.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00eb30 tx=0x7f1e1c00ee40 comp rx=0 tx=0).stop 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 shutdown_connections 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e100b2990 0x7f1e100b4e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e2c10fcb0 0x7f1e2c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e2c075970 0x7f1e2c19bd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 --2- 172.21.15.90:0/2469928842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e2c076290 0x7f1e2c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 >> 172.21.15.90:0/2469928842 conn(0x7f1e2c0fefc0 msgr2=0x7f1e2c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 shutdown_connections 2024-06-02T15:41:15.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.711+0000 7f1e2be00700 1 -- 172.21.15.90:0/2469928842 wait complete. 2024-06-02T15:41:16.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/662951885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T15:41:16.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2469928842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 -- 172.21.15.90:0/738285775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb940104630 msgr2=0x7fb940104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 --2- 172.21.15.90:0/738285775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb940104630 0x7fb940104a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098f0 tx=0x7fb930009c00 comp rx=0 tx=0).stop 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 -- 172.21.15.90:0/738285775 shutdown_connections 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 --2- 172.21.15.90:0/738285775 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb94010da60 0x7fb94010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 --2- 172.21.15.90:0/738285775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb940104630 0x7fb940104a10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 --2- 172.21.15.90:0/738285775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 -- 172.21.15.90:0/738285775 >> 172.21.15.90:0/738285775 conn(0x7fb9400fde60 msgr2=0x7fb940100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.350+0000 7fb93fe00700 1 -- 172.21.15.90:0/738285775 shutdown_connections 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.351+0000 7fb93fe00700 1 -- 172.21.15.90:0/738285775 wait complete. 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 Processor -- start 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 -- start start 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 0x7fb94019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94010da60 0x7fb9401a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb940113650 con 0x7fb940104f50 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401134d0 con 0x7fb940104630 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.352+0000 7fb93fe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9401137d0 con 0x7fb94010da60 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 0x7fb94019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42674/0 (socket says 172.21.15.90:42674) 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93e000700 1 -- 172.21.15.90:0/662951885 learned_addr learned my addr 172.21.15.90:0/662951885 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93f400700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94010da60 0x7fb9401a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 -- 172.21.15.90:0/662951885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94010da60 msgr2=0x7fb9401a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94010da60 0x7fb9401a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 -- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 msgr2=0x7fb94019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 -- 172.21.15.90:0/662951885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9401a0db0 con 0x7fb940104630 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.353+0000 7fb93ea00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 0x7fb94019bda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb930009230 tx=0x7fb93000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.354+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930005040 con 0x7fb940104630 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.354+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9401a1040 con 0x7fb940104630 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.354+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9401a8920 con 0x7fb940104630 2024-06-02T15:41:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.354+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb930005690 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.354+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93000b560 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.355+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb920005330 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.357+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb930003680 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.359+0000 7fb937400700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 0x7fb9240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.359+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb9300e3510 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.359+0000 7fb93e000700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 0x7fb9240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.360+0000 7fb93e000700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 0x7fb9240b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb9401042b0 tx=0x7fb92c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.361+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb930013030 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.623+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb9200058f0 con 0x7fb940104630 2024-06-02T15:41:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.624+0000 7fb937400700 1 -- 172.21.15.90:0/662951885 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9300aad50 con 0x7fb940104630 2024-06-02T15:41:16.518 INFO:teuthology.orchestra.run.smithi090.stdout:137438953565 2024-06-02T15:41:16.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 msgr2=0x7fb9240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:16.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 0x7fb9240b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb9401042b0 tx=0x7fb92c009260 comp rx=0 tx=0).stop 2024-06-02T15:41:16.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 msgr2=0x7fb94019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:16.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 0x7fb94019bda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb930009230 tx=0x7fb93000fa00 comp rx=0 tx=0).stop 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 shutdown_connections 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb9240b2910 0x7fb9240b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94010da60 0x7fb9401a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb940104630 0x7fb94019bda0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 --2- 172.21.15.90:0/662951885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb940104f50 0x7fb94019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.628+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 >> 172.21.15.90:0/662951885 conn(0x7fb9400fde60 msgr2=0x7fb9400fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.629+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 shutdown_connections 2024-06-02T15:41:16.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:15.629+0000 7fb93fe00700 1 -- 172.21.15.90:0/662951885 wait complete. 2024-06-02T15:41:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:16 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/662951885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T15:41:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:16 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2469928842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T15:41:16.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:16 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/662951885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T15:41:16.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:16 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2469928842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T15:41:17.410 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953562 got 137438953565 for osd.4 2024-06-02T15:41:17.410 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:17 smithi120 ceph-mon[36451]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-02T15:41:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:17 smithi135 ceph-mon[36839]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-02T15:41:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:17 smithi090 ceph-mon[31595]: pgmap v419: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-06-02T15:41:18.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411431 got 77309411434 for osd.2 2024-06-02T15:41:18.357 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:19 smithi120 ceph-mon[36451]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:19 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:41:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:19 smithi135 ceph-mon[36839]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:19 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:41:19.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:19 smithi090 ceph-mon[31595]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:19.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:19 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:41:20.208 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149789 got 111669149793 for osd.3 2024-06-02T15:41:20.208 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:20.512 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:20 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:20 smithi090 ceph-mon[31595]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:20 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:20 smithi120 ceph-mon[36451]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:20 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:20.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:20 smithi135 ceph-mon[36839]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:21.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:41:21.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:41:21.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:22.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:41:22.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:41:22.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:22.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:41:22.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:41:22.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:41:22.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 -- 172.21.15.90:0/3553666374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8075770 msgr2=0x7fa6d8075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 --2- 172.21.15.90:0/3553666374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8075770 0x7fa6d8075b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009990 tx=0x7fa6cc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 -- 172.21.15.90:0/3553666374 shutdown_connections 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 --2- 172.21.15.90:0/3553666374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8114710 0x7fa6d8116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 --2- 172.21.15.90:0/3553666374 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8075770 0x7fa6d8075b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 --2- 172.21.15.90:0/3553666374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8076090 0x7fa6d81141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 -- 172.21.15.90:0/3553666374 >> 172.21.15.90:0/3553666374 conn(0x7fa6d807a6f0 msgr2=0x7fa6d807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 -- 172.21.15.90:0/3553666374 shutdown_connections 2024-06-02T15:41:22.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.336+0000 7fa6e0600700 1 -- 172.21.15.90:0/3553666374 wait complete. 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 Processor -- start 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 -- start start 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 0x7fa6d81a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8114710 0x7fa6d81a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8119da0 con 0x7fa6d8075770 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8119c20 con 0x7fa6d8076090 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.337+0000 7fa6e0600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8119f20 con 0x7fa6d8114710 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6ddc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6ddc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46446/0 (socket says 172.21.15.90:46446) 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6ddc00700 1 -- 172.21.15.90:0/2503634402 learned_addr learned my addr 172.21.15.90:0/2503634402 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 0x7fa6d81a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6de600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8114710 0x7fa6d81a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 -- 172.21.15.90:0/2503634402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8114710 msgr2=0x7fa6d81a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8114710 0x7fa6d81a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 -- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 msgr2=0x7fa6d81a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6dd200700 1 -- 172.21.15.90:0/2503634402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d81a5280 con 0x7fa6d8076090 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.338+0000 7fa6ddc00700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6dd200700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 0x7fa6d81a07b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400e9f0 tx=0x7fa6c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4004220 con 0x7fa6d8076090 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c40043c0 con 0x7fa6d8076090 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d81a5570 con 0x7fa6d8076090 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4005020 con 0x7fa6d8076090 2024-06-02T15:41:22.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.339+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81acdf0 con 0x7fa6d8076090 2024-06-02T15:41:22.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.340+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d806a230 con 0x7fa6d8076090 2024-06-02T15:41:22.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.343+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa6c4004750 con 0x7fa6d8076090 2024-06-02T15:41:22.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.344+0000 7fa6d6000700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 0x7fa6c00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.344+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa6c40e27a0 con 0x7fa6d8076090 2024-06-02T15:41:22.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.344+0000 7fa6ddc00700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 0x7fa6c00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.345+0000 7fa6ddc00700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 0x7fa6c00b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006010 tx=0x7fa6cc00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:22.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.346+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6c40a9fe0 con 0x7fa6d8076090 2024-06-02T15:41:22.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.594+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa6d81a15d0 con 0x7fa6d8076090 2024-06-02T15:41:22.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.595+0000 7fa6d6000700 1 -- 172.21.15.90:0/2503634402 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6d81a15d0 con 0x7fa6d8076090 2024-06-02T15:41:22.593 INFO:teuthology.orchestra.run.smithi090.stdout:257698037825 2024-06-02T15:41:22.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 msgr2=0x7fa6c00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 0x7fa6c00b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc006010 tx=0x7fa6cc00b280 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 msgr2=0x7fa6d81a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 0x7fa6d81a07b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400e9f0 tx=0x7fa6c400ed00 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 shutdown_connections 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6c00b2990 0x7fa6c00b4e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d8114710 0x7fa6d81a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6d8075770 0x7fa6d81a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 --2- 172.21.15.90:0/2503634402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d8076090 0x7fa6d81a07b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 >> 172.21.15.90:0/2503634402 conn(0x7fa6d807a6f0 msgr2=0x7fa6d8113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 shutdown_connections 2024-06-02T15:41:22.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.598+0000 7fa6e0600700 1 -- 172.21.15.90:0/2503634402 wait complete. 2024-06-02T15:41:22.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 -- 172.21.15.90:0/256551873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 msgr2=0x7f853c075cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 --2- 172.21.15.90:0/256551873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c075cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f852c009990 tx=0x7f852c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 -- 172.21.15.90:0/256551873 shutdown_connections 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 --2- 172.21.15.90:0/256551873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 0x7f853c116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 --2- 172.21.15.90:0/256551873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c075cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 --2- 172.21.15.90:0/256551873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 0x7f853c113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 -- 172.21.15.90:0/256551873 >> 172.21.15.90:0/256551873 conn(0x7f853c0fefc0 msgr2=0x7f853c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 -- 172.21.15.90:0/256551873 shutdown_connections 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.246+0000 7f853b200700 1 -- 172.21.15.90:0/256551873 wait complete. 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.247+0000 7f853b200700 1 Processor -- start 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 -- start start 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 0x7f853c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 0x7f853c1a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f853c119c60 con 0x7f853c0758e0 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f853c119ae0 con 0x7f853c114480 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f853b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f853c119de0 con 0x7f853c076200 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f8539e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f8539e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46434/0 (socket says 172.21.15.90:46434) 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.248+0000 7f8539e00700 1 -- 172.21.15.90:0/3940707785 learned_addr learned my addr 172.21.15.90:0/3940707785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539400700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 0x7f853c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f853a800700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 0x7f853c1a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 -- 172.21.15.90:0/3940707785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 msgr2=0x7f853c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 0x7f853c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 -- 172.21.15.90:0/3940707785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 msgr2=0x7f853c1a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 0x7f853c1a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 -- 172.21.15.90:0/3940707785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c1a5140 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8539e00700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f852c0057f0 tx=0x7f852c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c026070 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c021410 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.249+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a53d0 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.250+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c017650 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.250+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1accb0 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.251+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c06a230 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.253+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f852c029070 con 0x7f853c0758e0 2024-06-02T15:41:22.604 INFO:teuthology.orchestra.run.smithi090.stdout:231928234056 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.254+0000 7f8532a00700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 0x7f85200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.254+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f852c0e3420 con 0x7f853c0758e0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.255+0000 7f8539400700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 0x7f85200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.255+0000 7f8539400700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 0x7f85200b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f853c107270 tx=0x7f8524009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.259+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f852c013070 con 0x7f853c0758e0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.513+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f853c02e560 con 0x7f853c0758e0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.513+0000 7f8532a00700 1 -- 172.21.15.90:0/3940707785 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f852c0aadd0 con 0x7f853c0758e0 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.516+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 msgr2=0x7f85200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.516+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 0x7f85200b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f853c107270 tx=0x7f8524009260 comp rx=0 tx=0).stop 2024-06-02T15:41:22.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.516+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 msgr2=0x7f853c1a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.516+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f852c0057f0 tx=0x7f852c00bfd0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 shutdown_connections 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85200b2990 0x7f85200b4e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f853c114480 0x7f853c1a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f853c0758e0 0x7f853c1a0130 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 --2- 172.21.15.90:0/3940707785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f853c076200 0x7f853c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 >> 172.21.15.90:0/3940707785 conn(0x7f853c0fefc0 msgr2=0x7f853c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 shutdown_connections 2024-06-02T15:41:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:22.517+0000 7f853b200700 1 -- 172.21.15.90:0/3940707785 wait complete. 2024-06-02T15:41:22.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:22 smithi090 ceph-mon[31595]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:22.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:22 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3940707785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T15:41:22.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:22 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2503634402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T15:41:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:22 smithi120 ceph-mon[36451]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:23.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:22 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3940707785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T15:41:23.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:22 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2503634402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T15:41:23.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:22 smithi135 ceph-mon[36839]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:23.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:22 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3940707785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T15:41:23.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:22 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2503634402' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T15:41:24.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234050 got 231928234056 for osd.8 2024-06-02T15:41:24.415 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.438+0000 7f9d03200700 1 -- 172.21.15.90:0/2794614544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc106140 msgr2=0x7f9cfc112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.438+0000 7f9d03200700 1 --2- 172.21.15.90:0/2794614544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc106140 0x7f9cfc112650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009f90 tx=0x7f9cf400b330 comp rx=0 tx=0).stop 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 -- 172.21.15.90:0/2794614544 shutdown_connections 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 --2- 172.21.15.90:0/2794614544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc106140 0x7f9cfc112650 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 --2- 172.21.15.90:0/2794614544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 --2- 172.21.15.90:0/2794614544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc105780 0x7f9cfc105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 -- 172.21.15.90:0/2794614544 >> 172.21.15.90:0/2794614544 conn(0x7f9cfc101270 msgr2=0x7f9cfc103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:24.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 -- 172.21.15.90:0/2794614544 shutdown_connections 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.439+0000 7f9d03200700 1 -- 172.21.15.90:0/2794614544 wait complete. 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 Processor -- start 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 -- start start 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 0x7f9cfc19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc115a10 con 0x7f9cfc10b780 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc115890 con 0x7f9cfc106140 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.440+0000 7f9d03200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc115b90 con 0x7f9cfc105780 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9d01200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 0x7f9cfc19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43486/0 (socket says 172.21.15.90:43486) 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9d01200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46468/0 (socket says 172.21.15.90:46468) 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfb400700 1 -- 172.21.15.90:0/1327976661 learned_addr learned my addr 172.21.15.90:0/1327976661 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 -- 172.21.15.90:0/1327976661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 msgr2=0x7f9cfc19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 -- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 msgr2=0x7f9cfc1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfbe00700 1 -- 172.21.15.90:0/1327976661 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc1a0de0 con 0x7f9cfc105780 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9cfb400700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.441+0000 7f9d01200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9cfbe00700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 0x7f9cfc19be20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00e9c0 tx=0x7f9cec00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00c9c0 con 0x7f9cfc105780 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc1a10d0 con 0x7f9cfc105780 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec00cb60 con 0x7f9cfc105780 2024-06-02T15:41:24.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec018420 con 0x7f9cfc105780 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.442+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1a8950 con 0x7f9cfc105780 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.443+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc06a230 con 0x7f9cfc105780 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.445+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9cec010430 con 0x7f9cfc105780 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.447+0000 7f9cf8c00700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 0x7f9ce00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.447+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9cec0e2d50 con 0x7f9cfc105780 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.447+0000 7f9cfb400700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 0x7f9ce00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.448+0000 7f9cfb400700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 0x7f9ce00b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc10acd0 tx=0x7f9ce8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:24.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.449+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cec0aa500 con 0x7f9cfc105780 2024-06-02T15:41:24.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.695+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9cfc02e560 con 0x7f9cfc105780 2024-06-02T15:41:24.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.696+0000 7f9cf8c00700 1 -- 172.21.15.90:0/1327976661 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9cec0a9e00 con 0x7f9cfc105780 2024-06-02T15:41:24.694 INFO:teuthology.orchestra.run.smithi090.stdout:184683593811 2024-06-02T15:41:24.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.699+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 msgr2=0x7f9ce00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.699+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 0x7f9ce00b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc10acd0 tx=0x7f9ce8009260 comp rx=0 tx=0).stop 2024-06-02T15:41:24.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.699+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 msgr2=0x7f9cfc19be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.699+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 0x7f9cfc19be20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00e9c0 tx=0x7f9cec00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 shutdown_connections 2024-06-02T15:41:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9ce00b2910 0x7f9ce00b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cfc10b780 0x7f9cfc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cfc105780 0x7f9cfc19be20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 --2- 172.21.15.90:0/1327976661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cfc106140 0x7f9cfc19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 >> 172.21.15.90:0/1327976661 conn(0x7f9cfc101270 msgr2=0x7f9cfc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 shutdown_connections 2024-06-02T15:41:24.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.700+0000 7f9d03200700 1 -- 172.21.15.90:0/1327976661 wait complete. 2024-06-02T15:41:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 -- 172.21.15.90:0/768185740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010e950 msgr2=0x7f61c010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 --2- 172.21.15.90:0/768185740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010e950 0x7f61c010ed30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009990 tx=0x7f61b0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 -- 172.21.15.90:0/768185740 shutdown_connections 2024-06-02T15:41:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 --2- 172.21.15.90:0/768185740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c010a440 0x7f61c01168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 --2- 172.21.15.90:0/768185740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010e950 0x7f61c010ed30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 --2- 172.21.15.90:0/768185740 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c0109a80 0x7f61c0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 -- 172.21.15.90:0/768185740 >> 172.21.15.90:0/768185740 conn(0x7f61c00792d0 msgr2=0x7f61c0108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 -- 172.21.15.90:0/768185740 shutdown_connections 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.732+0000 7f61c5200700 1 -- 172.21.15.90:0/768185740 wait complete. 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.733+0000 7f61c5200700 1 Processor -- start 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.733+0000 7f61c5200700 1 -- start start 2024-06-02T15:41:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c0109a80 0x7f61c01a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c010e950 0x7f61c01a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0119c60 con 0x7f61c010a440 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0119ae0 con 0x7f61c010e950 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61c5200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0119de0 con 0x7f61c0109a80 2024-06-02T15:41:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46496/0 (socket says 172.21.15.90:46496) 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 -- 172.21.15.90:0/2933645142 learned_addr learned my addr 172.21.15.90:0/2933645142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bf400700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c0109a80 0x7f61c01a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bfe00700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c010e950 0x7f61c01a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 -- 172.21.15.90:0/2933645142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c0109a80 msgr2=0x7f61c01a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c0109a80 0x7f61c01a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 -- 172.21.15.90:0/2933645142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c010e950 msgr2=0x7f61c01a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c010e950 0x7f61c01a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.734+0000 7f61bea00700 1 -- 172.21.15.90:0/2933645142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01a5260 con 0x7f61c010a440 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61bea00700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00eb30 tx=0x7f61ac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00cbd0 con 0x7f61c010a440 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61ac0044d0 con 0x7f61c010a440 2024-06-02T15:41:24.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01a5550 con 0x7f61c010a440 2024-06-02T15:41:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac003e30 con 0x7f61c010a440 2024-06-02T15:41:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.735+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01acdd0 con 0x7f61c010a440 2024-06-02T15:41:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.736+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c006a230 con 0x7f61c010a440 2024-06-02T15:41:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.739+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f61ac004000 con 0x7f61c010a440 2024-06-02T15:41:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.742+0000 7f61b7e00700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 0x7f61a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.742+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f61ac0e2b40 con 0x7f61c010a440 2024-06-02T15:41:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.742+0000 7f61bf400700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 0x7f61a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.743+0000 7f61bf400700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 0x7f61a40b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b2e0 tx=0x7f61b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.746+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61ac0042f0 con 0x7f61c010a440 2024-06-02T15:41:24.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:24 smithi090 ceph-mon[31595]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:24.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:24 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1327976661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T15:41:24.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.987+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f61c01a15b0 con 0x7f61c010a440 2024-06-02T15:41:24.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.988+0000 7f61b7e00700 1 -- 172.21.15.90:0/2933645142 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61ac0aa2f0 con 0x7f61c010a440 2024-06-02T15:41:24.986 INFO:teuthology.orchestra.run.smithi090.stdout:206158430285 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 msgr2=0x7f61a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 0x7f61a40b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b2e0 tx=0x7f61b0005fd0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 msgr2=0x7f61c01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00eb30 tx=0x7f61ac00ee40 comp rx=0 tx=0).stop 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 shutdown_connections 2024-06-02T15:41:24.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f61a40b2910 0x7f61a40b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f61c010e950 0x7f61c01a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61c0109a80 0x7f61c01a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 --2- 172.21.15.90:0/2933645142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61c010a440 0x7f61c01a0790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 >> 172.21.15.90:0/2933645142 conn(0x7f61c00792d0 msgr2=0x7f61c0079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 shutdown_connections 2024-06-02T15:41:24.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:24.991+0000 7f61c5200700 1 -- 172.21.15.90:0/2933645142 wait complete. 2024-06-02T15:41:25.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:24 smithi120 ceph-mon[36451]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:25.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:24 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1327976661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T15:41:25.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:24 smithi135 ceph-mon[36839]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:25.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:24 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1327976661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T15:41:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:25 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2933645142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T15:41:26.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:25 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2933645142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T15:41:26.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:25 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2933645142' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T15:41:26.950 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593806 got 184683593811 for osd.6 2024-06-02T15:41:26.953 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:27.055 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:27.057 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:26 smithi090 ceph-mon[31595]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:26 smithi120 ceph-mon[36451]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:27.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:26 smithi135 ceph-mon[36839]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:27.790 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037820 got 257698037825 for osd.9 2024-06-02T15:41:27.790 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:28.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430279 got 206158430285 for osd.7 2024-06-02T15:41:28.135 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:29.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:28 smithi090 ceph-mon[31595]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:29.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:28 smithi120 ceph-mon[36451]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:29.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:28 smithi135 ceph-mon[36839]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:29.982 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:29 smithi090 ceph-mon[31595]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-02T15:41:29.982 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:29 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3356785066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T15:41:30.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.791+0000 7f2c60e00700 1 -- 172.21.15.90:0/1797681431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c114460 msgr2=0x7f2c5c116850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:30.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.791+0000 7f2c60e00700 1 --2- 172.21.15.90:0/1797681431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c114460 0x7f2c5c116850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009f90 tx=0x7f2c5400b330 comp rx=0 tx=0).stop 2024-06-02T15:41:30.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 -- 172.21.15.90:0/1797681431 shutdown_connections 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 --2- 172.21.15.90:0/1797681431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c114460 0x7f2c5c116850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 --2- 172.21.15.90:0/1797681431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c0758e0 0x7f2c5c075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 --2- 172.21.15.90:0/1797681431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c076200 0x7f2c5c113f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 -- 172.21.15.90:0/1797681431 >> 172.21.15.90:0/1797681431 conn(0x7f2c5c0fef60 msgr2=0x7f2c5c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 -- 172.21.15.90:0/1797681431 shutdown_connections 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.792+0000 7f2c60e00700 1 -- 172.21.15.90:0/1797681431 wait complete. 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 Processor -- start 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 -- start start 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c0758e0 0x7f2c5c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c114460 0x7f2c5c1a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:30.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c119c80 con 0x7f2c5c114460 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c119b00 con 0x7f2c5c076200 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.793+0000 7f2c60e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c119e00 con 0x7f2c5c0758e0 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c114460 0x7f2c5c1a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43524/0 (socket says 172.21.15.90:43524) 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 -- 172.21.15.90:0/3356785066 learned_addr learned my addr 172.21.15.90:0/3356785066 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5b400700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c0758e0 0x7f2c5c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 -- 172.21.15.90:0/3356785066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c0758e0 msgr2=0x7f2c5c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c0758e0 0x7f2c5c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 -- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c114460 msgr2=0x7f2c5c1a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c114460 0x7f2c5c1a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 -- 172.21.15.90:0/3356785066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c1a51e0 con 0x7f2c5c076200 2024-06-02T15:41:30.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.794+0000 7f2c5aa00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9c0 tx=0x7f2c4800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.795+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800cbd0 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.795+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c4800ee50 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.795+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48005550 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.795+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c1a54d0 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.795+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c1acd50 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.796+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5c06a230 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.799+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2c48004750 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.801+0000 7f2c53e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 0x7f2c400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.801+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c480e2b70 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.801+0000 7f2c5b400700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 0x7f2c400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.802+0000 7f2c5b400700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 0x7f2c400b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005fd0 tx=0x7f2c4c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.802+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c48054df0 con 0x7f2c5c076200 2024-06-02T15:41:30.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:28.861+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c480a5eb0 con 0x7f2c5c076200 2024-06-02T15:41:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.050+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f2c5c02e560 con 0x7f2c5c076200 2024-06-02T15:41:30.213 INFO:teuthology.orchestra.run.smithi090.stdout:279172874301 2024-06-02T15:41:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.051+0000 7f2c53e00700 1 -- 172.21.15.90:0/3356785066 <== mon.2 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2c480b07c0 con 0x7f2c5c076200 2024-06-02T15:41:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 msgr2=0x7f2c400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:30.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 0x7f2c400b4e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005fd0 tx=0x7f2c4c009310 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 msgr2=0x7f2c5c1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e9c0 tx=0x7f2c4800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 shutdown_connections 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c400b2990 0x7f2c400b4e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c5c114460 0x7f2c5c1a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c0758e0 0x7f2c5c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 --2- 172.21.15.90:0/3356785066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2c5c076200 0x7f2c5c1a0710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 >> 172.21.15.90:0/3356785066 conn(0x7f2c5c0fef60 msgr2=0x7f2c5c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 shutdown_connections 2024-06-02T15:41:30.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:29.054+0000 7f2c60e00700 1 -- 172.21.15.90:0/3356785066 wait complete. 2024-06-02T15:41:30.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:29 smithi120 ceph-mon[36451]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-02T15:41:30.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:29 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3356785066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T15:41:30.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:29 smithi135 ceph-mon[36839]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-06-02T15:41:30.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:29 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3356785066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T15:41:31.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:30 smithi090 ceph-mon[31595]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:41:31.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:30 smithi120 ceph-mon[36451]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:41:31.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:30 smithi135 ceph-mon[36839]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:41:32.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 -- 172.21.15.90:0/1953633481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81037f0 msgr2=0x7fa1d8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:32.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 --2- 172.21.15.90:0/1953633481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81037f0 0x7fa1d8103bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0099c0 tx=0x7fa1cc009cd0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 -- 172.21.15.90:0/1953633481 shutdown_connections 2024-06-02T15:41:32.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 --2- 172.21.15.90:0/1953633481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81088f0 0x7fa1d8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 --2- 172.21.15.90:0/1953633481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81037f0 0x7fa1d8103bd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 --2- 172.21.15.90:0/1953633481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81041a0 0x7fa1d81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 -- 172.21.15.90:0/1953633481 >> 172.21.15.90:0/1953633481 conn(0x7fa1d807a6a0 msgr2=0x7fa1d807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.031+0000 7fa1e0200700 1 -- 172.21.15.90:0/1953633481 shutdown_connections 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.033+0000 7fa1e0200700 1 -- 172.21.15.90:0/1953633481 wait complete. 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 Processor -- start 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 -- start start 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 0x7fa1d819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 0x7fa1d819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8113710 con 0x7fa1d81088f0 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8113590 con 0x7fa1d81041a0 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.034+0000 7fa1e0200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8113890 con 0x7fa1d81037f0 2024-06-02T15:41:32.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1de200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1de200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40820/0 (socket says 172.21.15.90:40820) 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 0x7fa1d819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1de200700 1 -- 172.21.15.90:0/657035729 learned_addr learned my addr 172.21.15.90:0/657035729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dce00700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 0x7fa1d819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 -- 172.21.15.90:0/657035729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 msgr2=0x7fa1d819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 0x7fa1d819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 -- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 msgr2=0x7fa1d81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 -- 172.21.15.90:0/657035729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d81a0db0 con 0x7fa1d81037f0 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dce00700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 0x7fa1d819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1de200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.035+0000 7fa1dd800700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 0x7fa1d819bdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0098e0 tx=0x7fa1cc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.036+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc026070 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.036+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d81a1040 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.036+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1cc003c70 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.036+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00fa00 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.036+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a8920 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.037+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d806a230 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.040+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa1cc029070 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.041+0000 7fa1d6000700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 0x7fa1c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.041+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1cc0e2d10 con 0x7fa1d81037f0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.041+0000 7fa1dce00700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 0x7fa1c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.042+0000 7fa1dce00700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 0x7fa1c00b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1d81108b0 tx=0x7fa1c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:32.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.043+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1cc013070 con 0x7fa1d81037f0 2024-06-02T15:41:32.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874294 got 279172874301 for osd.10 2024-06-02T15:41:32.181 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:32.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.284+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa1d81a8af0 con 0x7fa1d81037f0 2024-06-02T15:41:32.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.285+0000 7fa1d6000700 1 -- 172.21.15.90:0/657035729 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1cc0aa6b0 con 0x7fa1d81037f0 2024-06-02T15:41:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:55834574962 2024-06-02T15:41:32.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 msgr2=0x7fa1c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:32.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 0x7fa1c00b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa1d81108b0 tx=0x7fa1c4009260 comp rx=0 tx=0).stop 2024-06-02T15:41:32.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 msgr2=0x7fa1d819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 0x7fa1d819bdf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0098e0 tx=0x7fa1cc004270 comp rx=0 tx=0).stop 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 shutdown_connections 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1c00b2900 0x7fa1c00b4dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1d81088f0 0x7fa1d81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1d81037f0 0x7fa1d819bdf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 --2- 172.21.15.90:0/657035729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1d81041a0 0x7fa1d819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.288+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 >> 172.21.15.90:0/657035729 conn(0x7fa1d807a6a0 msgr2=0x7fa1d81015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:32.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.289+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 shutdown_connections 2024-06-02T15:41:32.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:32.289+0000 7fa1e0200700 1 -- 172.21.15.90:0/657035729 wait complete. 2024-06-02T15:41:33.090 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:32 smithi090 ceph-mon[31595]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:41:33.090 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:32 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/657035729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T15:41:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:32 smithi120 ceph-mon[36451]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:41:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:32 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/657035729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T15:41:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:32 smithi135 ceph-mon[36839]: pgmap v427: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:41:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:32 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/657035729' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.023+0000 7f11eb200700 1 -- 172.21.15.90:0/168198430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec10b650 msgr2=0x7f11ec10ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.023+0000 7f11eb200700 1 --2- 172.21.15.90:0/168198430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec10b650 0x7f11ec10ba30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009990 tx=0x7f11dc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.023+0000 7f11eb200700 1 -- 172.21.15.90:0/168198430 shutdown_connections 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.023+0000 7f11eb200700 1 --2- 172.21.15.90:0/168198430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec106010 0x7f11ec112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.024+0000 7f11eb200700 1 --2- 172.21.15.90:0/168198430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec10b650 0x7f11ec10ba30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.024+0000 7f11eb200700 1 --2- 172.21.15.90:0/168198430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec105650 0x7f11ec105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.024+0000 7f11eb200700 1 -- 172.21.15.90:0/168198430 >> 172.21.15.90:0/168198430 conn(0x7f11ec101120 msgr2=0x7f11ec103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:34.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.024+0000 7f11eb200700 1 -- 172.21.15.90:0/168198430 shutdown_connections 2024-06-02T15:41:34.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.024+0000 7f11eb200700 1 -- 172.21.15.90:0/168198430 wait complete. 2024-06-02T15:41:34.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.025+0000 7f11eb200700 1 Processor -- start 2024-06-02T15:41:34.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.025+0000 7f11eb200700 1 -- start start 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.025+0000 7f11eb200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11eb200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec106010 0x7f11ec19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11eb200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 0x7f11ec1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11eb200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec1158a0 con 0x7f11ec105650 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11eb200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec115720 con 0x7f11ec10b650 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11eb200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec115a20 con 0x7f11ec106010 2024-06-02T15:41:34.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40834/0 (socket says 172.21.15.90:40834) 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 -- 172.21.15.90:0/771412239 learned_addr learned my addr 172.21.15.90:0/771412239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11ea800700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 0x7f11ec1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9400700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec106010 0x7f11ec19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 -- 172.21.15.90:0/771412239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec106010 msgr2=0x7f11ec19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec106010 0x7f11ec19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 -- 172.21.15.90:0/771412239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 msgr2=0x7f11ec1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 0x7f11ec1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11e9e00700 1 -- 172.21.15.90:0/771412239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1a0d80 con 0x7f11ec105650 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.026+0000 7f11ea800700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 0x7f11ec1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11e9e00700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc027070 con 0x7f11ec105650 2024-06-02T15:41:34.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11dc022470 con 0x7f11ec105650 2024-06-02T15:41:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1a1010 con 0x7f11ec105650 2024-06-02T15:41:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc017650 con 0x7f11ec105650 2024-06-02T15:41:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.027+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a88f0 con 0x7f11ec105650 2024-06-02T15:41:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.028+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec106d90 con 0x7f11ec105650 2024-06-02T15:41:34.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.030+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f11dc02a070 con 0x7f11ec105650 2024-06-02T15:41:34.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.032+0000 7f11e2a00700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 0x7f11d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:34.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.032+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f11dc0e32b0 con 0x7f11ec105650 2024-06-02T15:41:34.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.032+0000 7f11e9400700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 0x7f11d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:34.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.033+0000 7f11e9400700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 0x7f11d00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11ec103750 tx=0x7f11d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:34.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.036+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11dc013070 con 0x7f11ec105650 2024-06-02T15:41:34.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.263+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f11ec06a230 con 0x7f11ec105650 2024-06-02T15:41:34.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.264+0000 7f11e2a00700 1 -- 172.21.15.90:0/771412239 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f11dc0aac50 con 0x7f11ec105650 2024-06-02T15:41:34.262 INFO:teuthology.orchestra.run.smithi090.stdout:304942678074 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.266+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 msgr2=0x7f11d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.266+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 0x7f11d00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f11ec103750 tx=0x7f11d8009260 comp rx=0 tx=0).stop 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.266+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 msgr2=0x7f11ec19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.266+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000c00 tx=0x7f11dc0057c0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 shutdown_connections 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f11d00b2880 0x7f11d00b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11ec10b650 0x7f11ec1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11ec105650 0x7f11ec19bd70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 --2- 172.21.15.90:0/771412239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec106010 0x7f11ec19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 >> 172.21.15.90:0/771412239 conn(0x7f11ec101120 msgr2=0x7f11ec1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 shutdown_connections 2024-06-02T15:41:34.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:34.267+0000 7f11eb200700 1 -- 172.21.15.90:0/771412239 wait complete. 2024-06-02T15:41:34.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574954 got 55834574962 for osd.1 2024-06-02T15:41:34.577 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:34 smithi120 ceph-mon[36451]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:34 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/771412239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T15:41:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:34 smithi135 ceph-mon[36839]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:34 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/771412239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T15:41:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:34 smithi090 ceph-mon[31595]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:34 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/771412239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T15:41:36.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.171+0000 7fede5400700 1 -- 172.21.15.90:0/1915546849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010a780 msgr2=0x7fede010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:36.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.171+0000 7fede5400700 1 --2- 172.21.15.90:0/1915546849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010a780 0x7fede010e7d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009920 tx=0x7fedcc009c30 comp rx=0 tx=0).stop 2024-06-02T15:41:36.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 -- 172.21.15.90:0/1915546849 shutdown_connections 2024-06-02T15:41:36.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 --2- 172.21.15.90:0/1915546849 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede010eeb0 0x7fede0116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 --2- 172.21.15.90:0/1915546849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede0109dd0 0x7fede010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 --2- 172.21.15.90:0/1915546849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010a780 0x7fede010e7d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 -- 172.21.15.90:0/1915546849 >> 172.21.15.90:0/1915546849 conn(0x7fede0076d10 msgr2=0x7fede0079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:36.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 -- 172.21.15.90:0/1915546849 shutdown_connections 2024-06-02T15:41:36.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.172+0000 7fede5400700 1 -- 172.21.15.90:0/1915546849 wait complete. 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.173+0000 7fede5400700 1 Processor -- start 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.173+0000 7fede5400700 1 -- start start 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede010a780 0x7fede01a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010eeb0 0x7fede01a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fede0119d80 con 0x7fede010a780 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fede0119c00 con 0x7fede0109dd0 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fede5400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fede0119f00 con 0x7fede010eeb0 2024-06-02T15:41:36.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7feddde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede010a780 0x7fede01a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fedde800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fedde800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:44108/0 (socket says 172.21.15.90:44108) 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7fedde800700 1 -- 172.21.15.90:0/3616890377 learned_addr learned my addr 172.21.15.90:0/3616890377 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.174+0000 7feddf200700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010eeb0 0x7fede01a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 -- 172.21.15.90:0/3616890377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010eeb0 msgr2=0x7fede01a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010eeb0 0x7fede01a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 -- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede010a780 msgr2=0x7fede01a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede010a780 0x7fede01a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 -- 172.21.15.90:0/3616890377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede01a53b0 con 0x7fede0109dd0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedde800700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9a0 tx=0x7fedd000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd000ca10 con 0x7fede0109dd0 2024-06-02T15:41:36.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd000cbb0 con 0x7fede0109dd0 2024-06-02T15:41:36.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01a56a0 con 0x7fede0109dd0 2024-06-02T15:41:36.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.175+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0018680 con 0x7fede0109dd0 2024-06-02T15:41:36.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.176+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede01acf20 con 0x7fede0109dd0 2024-06-02T15:41:36.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.177+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede006a230 con 0x7fede0109dd0 2024-06-02T15:41:36.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.179+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fedd0010430 con 0x7fede0109dd0 2024-06-02T15:41:36.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.180+0000 7fedd7400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:36.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.181+0000 7feddde00700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:36.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.181+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fedd00e2790 con 0x7fede0109dd0 2024-06-02T15:41:36.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.181+0000 7feddde00700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 0x7fedc40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098f0 tx=0x7fedcc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:36.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.185+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fedd00aa0a0 con 0x7fede0109dd0 2024-06-02T15:41:36.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.418+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fede002e560 con 0x7fede0109dd0 2024-06-02T15:41:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.419+0000 7fedd7400700 1 -- 172.21.15.90:0/3616890377 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fedd00b2120 con 0x7fede0109dd0 2024-06-02T15:41:36.417 INFO:teuthology.orchestra.run.smithi090.stdout:158913790044 2024-06-02T15:41:36.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 msgr2=0x7fedc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:36.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 0x7fedc40b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098f0 tx=0x7fedcc019040 comp rx=0 tx=0).stop 2024-06-02T15:41:36.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 msgr2=0x7fede01a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:36.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9a0 tx=0x7fedd000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 shutdown_connections 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fedc40b2880 0x7fedc40b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fede010eeb0 0x7fede01a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fede0109dd0 0x7fede01a03a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 --2- 172.21.15.90:0/3616890377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fede010a780 0x7fede01a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 >> 172.21.15.90:0/3616890377 conn(0x7fede0076d10 msgr2=0x7fede00781b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 shutdown_connections 2024-06-02T15:41:36.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:36.422+0000 7fede5400700 1 -- 172.21.15.90:0/3616890377 wait complete. 2024-06-02T15:41:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:36 smithi090 ceph-mon[31595]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:36 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3616890377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T15:41:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:36 smithi120 ceph-mon[36451]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:36 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3616890377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T15:41:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:36 smithi135 ceph-mon[36839]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:36 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3616890377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T15:41:37.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678068 got 304942678074 for osd.11 2024-06-02T15:41:37.320 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:37.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790037 got 158913790044 for osd.5 2024-06-02T15:41:37.808 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 -- 172.21.15.90:0/1084391435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 msgr2=0x7fa40c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 --2- 172.21.15.90:0/1084391435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c116a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b0a0 tx=0x7fa40400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 -- 172.21.15.90:0/1084391435 shutdown_connections 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 --2- 172.21.15.90:0/1084391435 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c116a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 --2- 172.21.15.90:0/1084391435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c109dd0 0x7fa40c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 --2- 172.21.15.90:0/1084391435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c10a780 0x7fa40c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 -- 172.21.15.90:0/1084391435 >> 172.21.15.90:0/1084391435 conn(0x7fa40c076d10 msgr2=0x7fa40c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 -- 172.21.15.90:0/1084391435 shutdown_connections 2024-06-02T15:41:38.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.392+0000 7fa414400700 1 -- 172.21.15.90:0/1084391435 wait complete. 2024-06-02T15:41:38.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.393+0000 7fa414400700 1 Processor -- start 2024-06-02T15:41:38.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 -- start start 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 0x7fa40c1a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c1a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c119db0 con 0x7fa40c10a780 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c119c30 con 0x7fa40c10eeb0 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.394+0000 7fa414400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c119f30 con 0x7fa40c109dd0 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40858/0 (socket says 172.21.15.90:40858) 2024-06-02T15:41:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 -- 172.21.15.90:0/777160312 learned_addr learned my addr 172.21.15.90:0/777160312 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:38.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411a00700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 0x7fa40c1a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:38.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa412400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c1a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 -- 172.21.15.90:0/777160312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 msgr2=0x7fa40c1a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 0x7fa40c1a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 -- 172.21.15.90:0/777160312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 msgr2=0x7fa40c1a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c1a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411000700 1 -- 172.21.15.90:0/777160312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c1a5210 con 0x7fa40c10a780 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa412400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.395+0000 7fa411a00700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 0x7fa40c1a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa411000700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e9f0 tx=0x7fa3f800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004a30 con 0x7fa40c10a780 2024-06-02T15:41:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c1a5500 con 0x7fa40c10a780 2024-06-02T15:41:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f800cc70 con 0x7fa40c10a780 2024-06-02T15:41:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8005530 con 0x7fa40c10a780 2024-06-02T15:41:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.396+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c1acd80 con 0x7fa40c10a780 2024-06-02T15:41:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.397+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c06a230 con 0x7fa40c10a780 2024-06-02T15:41:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.399+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa3f8004bd0 con 0x7fa40c10a780 2024-06-02T15:41:38.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.402+0000 7fa40a000700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 0x7fa3f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:38.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.402+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa3f80e2720 con 0x7fa40c10a780 2024-06-02T15:41:38.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.402+0000 7fa411a00700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 0x7fa3f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:38.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.403+0000 7fa411a00700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 0x7fa3f40b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa400011490 tx=0x7fa400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:38.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.407+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3f800ce20 con 0x7fa40c10a780 2024-06-02T15:41:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.644+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa40c02e560 con 0x7fa40c10a780 2024-06-02T15:41:38.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.645+0000 7fa40a000700 1 -- 172.21.15.90:0/777160312 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa3f80aa030 con 0x7fa40c10a780 2024-06-02T15:41:38.643 INFO:teuthology.orchestra.run.smithi090.stdout:34359738488 2024-06-02T15:41:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 msgr2=0x7fa3f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 0x7fa3f40b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa400011490 tx=0x7fa400009260 comp rx=0 tx=0).stop 2024-06-02T15:41:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 msgr2=0x7fa40c1a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e9f0 tx=0x7fa3f800ed00 comp rx=0 tx=0).stop 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 shutdown_connections 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3f40b2900 0x7fa3f40b4dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa40c10eeb0 0x7fa40c1a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa40c109dd0 0x7fa40c1a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 --2- 172.21.15.90:0/777160312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa40c10a780 0x7fa40c1a0740 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 >> 172.21.15.90:0/777160312 conn(0x7fa40c076d10 msgr2=0x7fa40c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 shutdown_connections 2024-06-02T15:41:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:38.648+0000 7fa414400700 1 -- 172.21.15.90:0/777160312 wait complete. 2024-06-02T15:41:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:38 smithi120 ceph-mon[36451]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:38 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/777160312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T15:41:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:38 smithi135 ceph-mon[36839]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:38 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/777160312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T15:41:39.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:38 smithi090 ceph-mon[31595]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:39.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:38 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/777160312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T15:41:39.386 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738481 got 34359738488 for osd.0 2024-06-02T15:41:39.387 DEBUG:teuthology.parallel:result is None 2024-06-02T15:41:39.387 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-02T15:41:39.387 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T15:41:39.830 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:41.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:40 smithi090 ceph-mon[31595]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:40 smithi120 ceph-mon[36451]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:40 smithi135 ceph-mon[36839]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:41:41.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 -- 172.21.15.90:0/3916402212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e4109dd0 msgr2=0x7f36e410a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 --2- 172.21.15.90:0/3916402212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e4109dd0 0x7f36e410a1b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f36d80099c0 tx=0x7f36d8009cd0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 -- 172.21.15.90:0/3916402212 shutdown_connections 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 --2- 172.21.15.90:0/3916402212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e4116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 --2- 172.21.15.90:0/3916402212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e4109dd0 0x7f36e410a1b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 --2- 172.21.15.90:0/3916402212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e410a780 0x7f36e410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 -- 172.21.15.90:0/3916402212 >> 172.21.15.90:0/3916402212 conn(0x7f36e4076d10 msgr2=0x7f36e4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 -- 172.21.15.90:0/3916402212 shutdown_connections 2024-06-02T15:41:41.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.966+0000 7f36eb600700 1 -- 172.21.15.90:0/3916402212 wait complete. 2024-06-02T15:41:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.967+0000 7f36eb600700 1 Processor -- start 2024-06-02T15:41:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 -- start start 2024-06-02T15:41:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 0x7f36e41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:41.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e41a4de0 0x7f36e41a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:41.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4119da0 con 0x7f36e410a780 2024-06-02T15:41:41.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4119c20 con 0x7f36e41a4de0 2024-06-02T15:41:41.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36eb600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4119f20 con 0x7f36e410eeb0 2024-06-02T15:41:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 0x7f36e41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36e3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36e3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52556/0 (socket says 172.21.15.90:52556) 2024-06-02T15:41:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.968+0000 7f36e3e00700 1 -- 172.21.15.90:0/330531389 learned_addr learned my addr 172.21.15.90:0/330531389 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:41.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e9600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e41a4de0 0x7f36e41a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 -- 172.21.15.90:0/330531389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e41a4de0 msgr2=0x7f36e41a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e41a4de0 0x7f36e41a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 -- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 msgr2=0x7f36e41a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 0x7f36e41a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 -- 172.21.15.90:0/330531389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e41a5900 con 0x7f36e410eeb0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e8c00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 0x7f36e41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.969+0000 7f36e3e00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e970 tx=0x7f36d000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.970+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004a30 con 0x7f36e410eeb0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.970+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41a5bf0 con 0x7f36e410eeb0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.970+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d000cc90 con 0x7f36e410eeb0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.970+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0005500 con 0x7f36e410eeb0 2024-06-02T15:41:41.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.970+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41ad420 con 0x7f36e410eeb0 2024-06-02T15:41:41.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.971+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e406a230 con 0x7f36e410eeb0 2024-06-02T15:41:41.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.973+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f36d0004bd0 con 0x7f36e410eeb0 2024-06-02T15:41:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.974+0000 7f36e1600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 0x7f36cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.975+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f36d00e2790 con 0x7f36e410eeb0 2024-06-02T15:41:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.975+0000 7f36e8c00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 0x7f36cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:41.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.975+0000 7f36e8c00700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 0x7f36cc0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b580 tx=0x7f36d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:41.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:41.980+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36d00aa0a0 con 0x7f36e410eeb0 2024-06-02T15:41:42.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.205+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36e4063980 con 0x7f36cc0b2880 2024-06-02T15:41:42.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.215+0000 7f36e1600700 1 -- 172.21.15.90:0/330531389 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340483 (secure 0 0 0) 0x7f36e4063980 con 0x7f36cc0b2880 2024-06-02T15:41:42.215 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:41:42.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 msgr2=0x7f36cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:42.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 0x7f36cc0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b580 tx=0x7f36d8005fb0 comp rx=0 tx=0).stop 2024-06-02T15:41:42.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 msgr2=0x7f36e41a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:42.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e970 tx=0x7f36d000ec80 comp rx=0 tx=0).stop 2024-06-02T15:41:42.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 shutdown_connections 2024-06-02T15:41:42.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36cc0b2880 0x7f36cc0b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36e41a4de0 0x7f36e41a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e410a780 0x7f36e41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 --2- 172.21.15.90:0/330531389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e410eeb0 0x7f36e41a0aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 >> 172.21.15.90:0/330531389 conn(0x7f36e4076d10 msgr2=0x7f36e4110080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 shutdown_connections 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:42.223+0000 7f36eb600700 1 -- 172.21.15.90:0/330531389 wait complete. 2024-06-02T15:41:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:41:43.015 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-06-02T15:41:41.774147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659585,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":844,"num_read_kb":522,"num_write":305,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375824,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566000,"statfs":{"total":1151940427776,"available":1151555584000,"internally_reserved":0,"allocated":14532608,"data_stored":8172605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005649"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706782+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:41:27.706782+0000","last_peered":"2024-06-02T15:41:27.706782+0000","last_clean":"2024-06-02T15:41:27.706782+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:41:27.706782+0000","last_undegraded":"2024-06-02T15:41:27.706782+0000","last_fullsized":"2024-06-02T15:41:27.706782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703557+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:41:27.703557+0000","last_peered":"2024-06-02T15:41:27.703557+0000","last_clean":"2024-06-02T15:41:27.703557+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:41:27.703557+0000","last_undegraded":"2024-06-02T15:41:27.703557+0000","last_fullsized":"2024-06-02T15:41:27.703557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706429+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:41:27.706429+0000","last_peered":"2024-06-02T15:41:27.706429+0000","last_clean":"2024-06-02T15:41:27.706429+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:41:27.706429+0000","last_undegraded":"2024-06-02T15:41:27.706429+0000","last_fullsized":"2024-06-02T15:41:27.706429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692023+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:41:27.692023+0000","last_peered":"2024-06-02T15:41:27.692023+0000","last_clean":"2024-06-02T15:41:27.692023+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:41:27.692023+0000","last_undegraded":"2024-06-02T15:41:27.692023+0000","last_fullsized":"2024-06-02T15:41:27.692023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699088+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:41:27.699088+0000","last_peered":"2024-06-02T15:41:27.699088+0000","last_clean":"2024-06-02T15:41:27.699088+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:41:27.699088+0000","last_undegraded":"2024-06-02T15:41:27.699088+0000","last_fullsized":"2024-06-02T15:41:27.699088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697398+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:41:27.697398+0000","last_peered":"2024-06-02T15:41:27.697398+0000","last_clean":"2024-06-02T15:41:27.697398+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:41:27.697398+0000","last_undegraded":"2024-06-02T15:41:27.697398+0000","last_fullsized":"2024-06-02T15:41:27.697398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.700624+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:41:27.700624+0000","last_peered":"2024-06-02T15:41:27.700624+0000","last_clean":"2024-06-02T15:41:27.700624+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:41:27.700624+0000","last_undegraded":"2024-06-02T15:41:27.700624+0000","last_fullsized":"2024-06-02T15:41:27.700624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711513+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:41:27.711513+0000","last_peered":"2024-06-02T15:41:27.711513+0000","last_clean":"2024-06-02T15:41:27.711513+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:41:27.711513+0000","last_undegraded":"2024-06-02T15:41:27.711513+0000","last_fullsized":"2024-06-02T15:41:27.711513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692843+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:41:27.692843+0000","last_peered":"2024-06-02T15:41:27.692843+0000","last_clean":"2024-06-02T15:41:27.692843+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:41:27.692843+0000","last_undegraded":"2024-06-02T15:41:27.692843+0000","last_fullsized":"2024-06-02T15:41:27.692843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698540+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:41:27.698540+0000","last_peered":"2024-06-02T15:41:27.698540+0000","last_clean":"2024-06-02T15:41:27.698540+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:41:27.698540+0000","last_undegraded":"2024-06-02T15:41:27.698540+0000","last_fullsized":"2024-06-02T15:41:27.698540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694241+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:41:27.694241+0000","last_peered":"2024-06-02T15:41:27.694241+0000","last_clean":"2024-06-02T15:41:27.694241+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:41:27.694241+0000","last_undegraded":"2024-06-02T15:41:27.694241+0000","last_fullsized":"2024-06-02T15:41:27.694241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690004+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:41:27.690004+0000","last_peered":"2024-06-02T15:41:27.690004+0000","last_clean":"2024-06-02T15:41:27.690004+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:41:27.690004+0000","last_undegraded":"2024-06-02T15:41:27.690004+0000","last_fullsized":"2024-06-02T15:41:27.690004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694651+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:41:27.694651+0000","last_peered":"2024-06-02T15:41:27.694651+0000","last_clean":"2024-06-02T15:41:27.694651+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:41:27.694651+0000","last_undegraded":"2024-06-02T15:41:27.694651+0000","last_fullsized":"2024-06-02T15:41:27.694651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699405+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:41:27.699405+0000","last_peered":"2024-06-02T15:41:27.699405+0000","last_clean":"2024-06-02T15:41:27.699405+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:41:27.699405+0000","last_undegraded":"2024-06-02T15:41:27.699405+0000","last_fullsized":"2024-06-02T15:41:27.699405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705584+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:41:27.705584+0000","last_peered":"2024-06-02T15:41:27.705584+0000","last_clean":"2024-06-02T15:41:27.705584+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:41:27.705584+0000","last_undegraded":"2024-06-02T15:41:27.705584+0000","last_fullsized":"2024-06-02T15:41:27.705584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702871+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:41:27.702871+0000","last_peered":"2024-06-02T15:41:27.702871+0000","last_clean":"2024-06-02T15:41:27.702871+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:41:27.702871+0000","last_undegraded":"2024-06-02T15:41:27.702871+0000","last_fullsized":"2024-06-02T15:41:27.702871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706832+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:41:27.706832+0000","last_peered":"2024-06-02T15:41:27.706832+0000","last_clean":"2024-06-02T15:41:27.706832+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:41:27.706832+0000","last_undegraded":"2024-06-02T15:41:27.706832+0000","last_fullsized":"2024-06-02T15:41:27.706832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701107+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:41:27.701107+0000","last_peered":"2024-06-02T15:41:27.701107+0000","last_clean":"2024-06-02T15:41:27.701107+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:41:27.701107+0000","last_undegraded":"2024-06-02T15:41:27.701107+0000","last_fullsized":"2024-06-02T15:41:27.701107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687376+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:41:27.687376+0000","last_peered":"2024-06-02T15:41:27.687376+0000","last_clean":"2024-06-02T15:41:27.687376+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:41:27.687376+0000","last_undegraded":"2024-06-02T15:41:27.687376+0000","last_fullsized":"2024-06-02T15:41:27.687376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030556+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:00.030556+0000","last_peered":"2024-06-02T15:40:00.030556+0000","last_clean":"2024-06-02T15:40:00.030556+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:00.030556+0000","last_undegraded":"2024-06-02T15:40:00.030556+0000","last_fullsized":"2024-06-02T15:40:00.030556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":67,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036770+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:00.036770+0000","last_peered":"2024-06-02T15:40:00.036770+0000","last_clean":"2024-06-02T15:40:00.036770+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:00.036770+0000","last_undegraded":"2024-06-02T15:40:00.036770+0000","last_fullsized":"2024-06-02T15:40:00.036770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702317+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:41:27.702317+0000","last_peered":"2024-06-02T15:41:27.702317+0000","last_clean":"2024-06-02T15:41:27.702317+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:41:27.702317+0000","last_undegraded":"2024-06-02T15:41:27.702317+0000","last_fullsized":"2024-06-02T15:41:27.702317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.042403+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:00.042403+0000","last_peered":"2024-06-02T15:40:00.042403+0000","last_clean":"2024-06-02T15:40:00.042403+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:00.042403+0000","last_undegraded":"2024-06-02T15:40:00.042403+0000","last_fullsized":"2024-06-02T15:40:00.042403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034821+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:00.034821+0000","last_peered":"2024-06-02T15:40:00.034821+0000","last_clean":"2024-06-02T15:40:00.034821+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:00.034821+0000","last_undegraded":"2024-06-02T15:40:00.034821+0000","last_fullsized":"2024-06-02T15:40:00.034821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708333+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:41:27.708333+0000","last_peered":"2024-06-02T15:41:27.708333+0000","last_clean":"2024-06-02T15:41:27.708333+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:41:27.708333+0000","last_undegraded":"2024-06-02T15:41:27.708333+0000","last_fullsized":"2024-06-02T15:41:27.708333+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031438+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031438+0000","last_peered":"2024-06-02T15:40:00.031438+0000","last_clean":"2024-06-02T15:40:00.031438+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031438+0000","last_undegraded":"2024-06-02T15:40:00.031438+0000","last_fullsized":"2024-06-02T15:40:00.031438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032580+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:00.032580+0000","last_peered":"2024-06-02T15:40:00.032580+0000","last_clean":"2024-06-02T15:40:00.032580+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:00.032580+0000","last_undegraded":"2024-06-02T15:40:00.032580+0000","last_fullsized":"2024-06-02T15:40:00.032580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695456+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:41:27.695456+0000","last_peered":"2024-06-02T15:41:27.695456+0000","last_clean":"2024-06-02T15:41:27.695456+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:41:27.695456+0000","last_undegraded":"2024-06-02T15:41:27.695456+0000","last_fullsized":"2024-06-02T15:41:27.695456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031342+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031342+0000","last_peered":"2024-06-02T15:40:00.031342+0000","last_clean":"2024-06-02T15:40:00.031342+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031342+0000","last_undegraded":"2024-06-02T15:40:00.031342+0000","last_fullsized":"2024-06-02T15:40:00.031342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032456+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:00.032456+0000","last_peered":"2024-06-02T15:40:00.032456+0000","last_clean":"2024-06-02T15:40:00.032456+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:00.032456+0000","last_undegraded":"2024-06-02T15:40:00.032456+0000","last_fullsized":"2024-06-02T15:40:00.032456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696093+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:41:27.696093+0000","last_peered":"2024-06-02T15:41:27.696093+0000","last_clean":"2024-06-02T15:41:27.696093+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:41:27.696093+0000","last_undegraded":"2024-06-02T15:41:27.696093+0000","last_fullsized":"2024-06-02T15:41:27.696093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031260+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031260+0000","last_peered":"2024-06-02T15:40:00.031260+0000","last_clean":"2024-06-02T15:40:00.031260+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031260+0000","last_undegraded":"2024-06-02T15:40:00.031260+0000","last_fullsized":"2024-06-02T15:40:00.031260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:37.322751+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:41:37.322751+0000","last_peered":"2024-06-02T15:41:37.322751+0000","last_clean":"2024-06-02T15:41:37.322751+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:41:37.322751+0000","last_undegraded":"2024-06-02T15:41:37.322751+0000","last_fullsized":"2024-06-02T15:41:37.322751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697520+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:41:27.697520+0000","last_peered":"2024-06-02T15:41:27.697520+0000","last_clean":"2024-06-02T15:41:27.697520+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:41:27.697520+0000","last_undegraded":"2024-06-02T15:41:27.697520+0000","last_fullsized":"2024-06-02T15:41:27.697520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031151+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031151+0000","last_peered":"2024-06-02T15:40:00.031151+0000","last_clean":"2024-06-02T15:40:00.031151+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031151+0000","last_undegraded":"2024-06-02T15:40:00.031151+0000","last_fullsized":"2024-06-02T15:40:00.031151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031630+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:00.031630+0000","last_peered":"2024-06-02T15:40:00.031630+0000","last_clean":"2024-06-02T15:40:00.031630+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:00.031630+0000","last_undegraded":"2024-06-02T15:40:00.031630+0000","last_fullsized":"2024-06-02T15:40:00.031630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711290+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:41:27.711290+0000","last_peered":"2024-06-02T15:41:27.711290+0000","last_clean":"2024-06-02T15:41:27.711290+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:41:27.711290+0000","last_undegraded":"2024-06-02T15:41:27.711290+0000","last_fullsized":"2024-06-02T15:41:27.711290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031144+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:00.031144+0000","last_peered":"2024-06-02T15:40:00.031144+0000","last_clean":"2024-06-02T15:40:00.031144+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:00.031144+0000","last_undegraded":"2024-06-02T15:40:00.031144+0000","last_fullsized":"2024-06-02T15:40:00.031144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033997+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:00.033997+0000","last_peered":"2024-06-02T15:40:00.033997+0000","last_clean":"2024-06-02T15:40:00.033997+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:00.033997+0000","last_undegraded":"2024-06-02T15:40:00.033997+0000","last_fullsized":"2024-06-02T15:40:00.033997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711334+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:41:27.711334+0000","last_peered":"2024-06-02T15:41:27.711334+0000","last_clean":"2024-06-02T15:41:27.711334+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:41:27.711334+0000","last_undegraded":"2024-06-02T15:41:27.711334+0000","last_fullsized":"2024-06-02T15:41:27.711334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026122+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:00.026122+0000","last_peered":"2024-06-02T15:40:00.026122+0000","last_clean":"2024-06-02T15:40:00.026122+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:00.026122+0000","last_undegraded":"2024-06-02T15:40:00.026122+0000","last_fullsized":"2024-06-02T15:40:00.026122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023805+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:00.023805+0000","last_peered":"2024-06-02T15:40:00.023805+0000","last_clean":"2024-06-02T15:40:00.023805+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:00.023805+0000","last_undegraded":"2024-06-02T15:40:00.023805+0000","last_fullsized":"2024-06-02T15:40:00.023805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710271+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:41:27.710271+0000","last_peered":"2024-06-02T15:41:27.710271+0000","last_clean":"2024-06-02T15:41:27.710271+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:41:27.710271+0000","last_undegraded":"2024-06-02T15:41:27.710271+0000","last_fullsized":"2024-06-02T15:41:27.710271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032043+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:00.032043+0000","last_peered":"2024-06-02T15:40:00.032043+0000","last_clean":"2024-06-02T15:40:00.032043+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:00.032043+0000","last_undegraded":"2024-06-02T15:40:00.032043+0000","last_fullsized":"2024-06-02T15:40:00.032043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":171,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:36.320736+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:41:36.320736+0000","last_peered":"2024-06-02T15:41:36.320736+0000","last_clean":"2024-06-02T15:41:36.320736+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:41:36.320736+0000","last_undegraded":"2024-06-02T15:41:36.320736+0000","last_fullsized":"2024-06-02T15:41:36.320736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701017+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:41:27.701017+0000","last_peered":"2024-06-02T15:41:27.701017+0000","last_clean":"2024-06-02T15:41:27.701017+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:41:27.701017+0000","last_undegraded":"2024-06-02T15:41:27.701017+0000","last_fullsized":"2024-06-02T15:41:27.701017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032926+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:00.032926+0000","last_peered":"2024-06-02T15:40:00.032926+0000","last_clean":"2024-06-02T15:40:00.032926+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:00.032926+0000","last_undegraded":"2024-06-02T15:40:00.032926+0000","last_fullsized":"2024-06-02T15:40:00.032926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027472+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:00.027472+0000","last_peered":"2024-06-02T15:40:00.027472+0000","last_clean":"2024-06-02T15:40:00.027472+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:00.027472+0000","last_undegraded":"2024-06-02T15:40:00.027472+0000","last_fullsized":"2024-06-02T15:40:00.027472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706776+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:41:27.706776+0000","last_peered":"2024-06-02T15:41:27.706776+0000","last_clean":"2024-06-02T15:41:27.706776+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:41:27.706776+0000","last_undegraded":"2024-06-02T15:41:27.706776+0000","last_fullsized":"2024-06-02T15:41:27.706776+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031950+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:00.031950+0000","last_peered":"2024-06-02T15:40:00.031950+0000","last_clean":"2024-06-02T15:40:00.031950+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:00.031950+0000","last_undegraded":"2024-06-02T15:40:00.031950+0000","last_fullsized":"2024-06-02T15:40:00.031950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024224+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:00.024224+0000","last_peered":"2024-06-02T15:40:00.024224+0000","last_clean":"2024-06-02T15:40:00.024224+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:00.024224+0000","last_undegraded":"2024-06-02T15:40:00.024224+0000","last_fullsized":"2024-06-02T15:40:00.024224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698155+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:41:27.698155+0000","last_peered":"2024-06-02T15:41:27.698155+0000","last_clean":"2024-06-02T15:41:27.698155+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:41:27.698155+0000","last_undegraded":"2024-06-02T15:41:27.698155+0000","last_fullsized":"2024-06-02T15:41:27.698155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034406+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:00.034406+0000","last_peered":"2024-06-02T15:40:00.034406+0000","last_clean":"2024-06-02T15:40:00.034406+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:00.034406+0000","last_undegraded":"2024-06-02T15:40:00.034406+0000","last_fullsized":"2024-06-02T15:40:00.034406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":352,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:37.322121+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:41:37.322121+0000","last_peered":"2024-06-02T15:41:37.322121+0000","last_clean":"2024-06-02T15:41:37.322121+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:41:37.322121+0000","last_undegraded":"2024-06-02T15:41:37.322121+0000","last_fullsized":"2024-06-02T15:41:37.322121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":254,"num_read_kb":190,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685689+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:41:27.685689+0000","last_peered":"2024-06-02T15:41:27.685689+0000","last_clean":"2024-06-02T15:41:27.685689+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:41:27.685689+0000","last_undegraded":"2024-06-02T15:41:27.685689+0000","last_fullsized":"2024-06-02T15:41:27.685689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024687+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:00.024687+0000","last_peered":"2024-06-02T15:40:00.024687+0000","last_clean":"2024-06-02T15:40:00.024687+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:00.024687+0000","last_undegraded":"2024-06-02T15:40:00.024687+0000","last_fullsized":"2024-06-02T15:40:00.024687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030857+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:00.030857+0000","last_peered":"2024-06-02T15:40:00.030857+0000","last_clean":"2024-06-02T15:40:00.030857+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:00.030857+0000","last_undegraded":"2024-06-02T15:40:00.030857+0000","last_fullsized":"2024-06-02T15:40:00.030857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710858+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:41:27.710858+0000","last_peered":"2024-06-02T15:41:27.710858+0000","last_clean":"2024-06-02T15:41:27.710858+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:41:27.710858+0000","last_undegraded":"2024-06-02T15:41:27.710858+0000","last_fullsized":"2024-06-02T15:41:27.710858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030018+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:00.030018+0000","last_peered":"2024-06-02T15:40:00.030018+0000","last_clean":"2024-06-02T15:40:00.030018+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:00.030018+0000","last_undegraded":"2024-06-02T15:40:00.030018+0000","last_fullsized":"2024-06-02T15:40:00.030018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024214+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:00.024214+0000","last_peered":"2024-06-02T15:40:00.024214+0000","last_clean":"2024-06-02T15:40:00.024214+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:00.024214+0000","last_undegraded":"2024-06-02T15:40:00.024214+0000","last_fullsized":"2024-06-02T15:40:00.024214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696457+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:41:27.696457+0000","last_peered":"2024-06-02T15:41:27.696457+0000","last_clean":"2024-06-02T15:41:27.696457+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:41:27.696457+0000","last_undegraded":"2024-06-02T15:41:27.696457+0000","last_fullsized":"2024-06-02T15:41:27.696457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036506+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:00.036506+0000","last_peered":"2024-06-02T15:40:00.036506+0000","last_clean":"2024-06-02T15:40:00.036506+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:00.036506+0000","last_undegraded":"2024-06-02T15:40:00.036506+0000","last_fullsized":"2024-06-02T15:40:00.036506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":177,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:37.388521+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:41:37.388521+0000","last_peered":"2024-06-02T15:41:37.388521+0000","last_clean":"2024-06-02T15:41:37.388521+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:41:37.388521+0000","last_undegraded":"2024-06-02T15:41:37.388521+0000","last_fullsized":"2024-06-02T15:41:37.388521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687441+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:41:27.687441+0000","last_peered":"2024-06-02T15:41:27.687441+0000","last_clean":"2024-06-02T15:41:27.687441+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:41:27.687441+0000","last_undegraded":"2024-06-02T15:41:27.687441+0000","last_fullsized":"2024-06-02T15:41:27.687441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038360+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:00.038360+0000","last_peered":"2024-06-02T15:40:00.038360+0000","last_clean":"2024-06-02T15:40:00.038360+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:00.038360+0000","last_undegraded":"2024-06-02T15:40:00.038360+0000","last_fullsized":"2024-06-02T15:40:00.038360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032025+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:00.032025+0000","last_peered":"2024-06-02T15:40:00.032025+0000","last_clean":"2024-06-02T15:40:00.032025+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:00.032025+0000","last_undegraded":"2024-06-02T15:40:00.032025+0000","last_fullsized":"2024-06-02T15:40:00.032025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694331+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:41:27.694331+0000","last_peered":"2024-06-02T15:41:27.694331+0000","last_clean":"2024-06-02T15:41:27.694331+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:41:27.694331+0000","last_undegraded":"2024-06-02T15:41:27.694331+0000","last_fullsized":"2024-06-02T15:41:27.694331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024046+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:00.024046+0000","last_peered":"2024-06-02T15:40:00.024046+0000","last_clean":"2024-06-02T15:40:00.024046+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:00.024046+0000","last_undegraded":"2024-06-02T15:40:00.024046+0000","last_fullsized":"2024-06-02T15:40:00.024046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032813+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:00.032813+0000","last_peered":"2024-06-02T15:40:00.032813+0000","last_clean":"2024-06-02T15:40:00.032813+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:00.032813+0000","last_undegraded":"2024-06-02T15:40:00.032813+0000","last_fullsized":"2024-06-02T15:40:00.032813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695201+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:41:27.695201+0000","last_peered":"2024-06-02T15:41:27.695201+0000","last_clean":"2024-06-02T15:41:27.695201+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:41:27.695201+0000","last_undegraded":"2024-06-02T15:41:27.695201+0000","last_fullsized":"2024-06-02T15:41:27.695201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032119+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:00.032119+0000","last_peered":"2024-06-02T15:40:00.032119+0000","last_clean":"2024-06-02T15:40:00.032119+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:00.032119+0000","last_undegraded":"2024-06-02T15:40:00.032119+0000","last_fullsized":"2024-06-02T15:40:00.032119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035913+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:00.035913+0000","last_peered":"2024-06-02T15:40:00.035913+0000","last_clean":"2024-06-02T15:40:00.035913+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:00.035913+0000","last_undegraded":"2024-06-02T15:40:00.035913+0000","last_fullsized":"2024-06-02T15:40:00.035913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693160+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:41:27.693160+0000","last_peered":"2024-06-02T15:41:27.693160+0000","last_clean":"2024-06-02T15:41:27.693160+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:41:27.693160+0000","last_undegraded":"2024-06-02T15:41:27.693160+0000","last_fullsized":"2024-06-02T15:41:27.693160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034575+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:00.034575+0000","last_peered":"2024-06-02T15:40:00.034575+0000","last_clean":"2024-06-02T15:40:00.034575+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:00.034575+0000","last_undegraded":"2024-06-02T15:40:00.034575+0000","last_fullsized":"2024-06-02T15:40:00.034575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026934+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:00.026934+0000","last_peered":"2024-06-02T15:40:00.026934+0000","last_clean":"2024-06-02T15:40:00.026934+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:00.026934+0000","last_undegraded":"2024-06-02T15:40:00.026934+0000","last_fullsized":"2024-06-02T15:40:00.026934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.676816+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:41:27.676816+0000","last_peered":"2024-06-02T15:41:27.676816+0000","last_clean":"2024-06-02T15:41:27.676816+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:41:27.676816+0000","last_undegraded":"2024-06-02T15:41:27.676816+0000","last_fullsized":"2024-06-02T15:41:27.676816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.042186+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:00.042186+0000","last_peered":"2024-06-02T15:40:00.042186+0000","last_clean":"2024-06-02T15:40:00.042186+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:00.042186+0000","last_undegraded":"2024-06-02T15:40:00.042186+0000","last_fullsized":"2024-06-02T15:40:00.042186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686863+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:41:27.686863+0000","last_peered":"2024-06-02T15:41:27.686863+0000","last_clean":"2024-06-02T15:41:27.686863+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:41:27.686863+0000","last_undegraded":"2024-06-02T15:41:27.686863+0000","last_fullsized":"2024-06-02T15:41:27.686863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":174,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:38.390208+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:41:38.390208+0000","last_peered":"2024-06-02T15:41:38.390208+0000","last_clean":"2024-06-02T15:41:38.390208+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:41:38.390208+0000","last_undegraded":"2024-06-02T15:41:38.390208+0000","last_fullsized":"2024-06-02T15:41:38.390208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.045737+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:00.045737+0000","last_peered":"2024-06-02T15:40:00.045737+0000","last_clean":"2024-06-02T15:40:00.045737+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:00.045737+0000","last_undegraded":"2024-06-02T15:40:00.045737+0000","last_fullsized":"2024-06-02T15:40:00.045737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703766+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:41:27.703766+0000","last_peered":"2024-06-02T15:41:27.703766+0000","last_clean":"2024-06-02T15:41:27.703766+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:41:27.703766+0000","last_undegraded":"2024-06-02T15:41:27.703766+0000","last_fullsized":"2024-06-02T15:41:27.703766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031825+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:00.031825+0000","last_peered":"2024-06-02T15:40:00.031825+0000","last_clean":"2024-06-02T15:40:00.031825+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:00.031825+0000","last_undegraded":"2024-06-02T15:40:00.031825+0000","last_fullsized":"2024-06-02T15:40:00.031825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026616+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:40:00.026616+0000","last_peered":"2024-06-02T15:40:00.026616+0000","last_clean":"2024-06-02T15:40:00.026616+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:40:00.026616+0000","last_undegraded":"2024-06-02T15:40:00.026616+0000","last_fullsized":"2024-06-02T15:40:00.026616+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024519+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:00.024519+0000","last_peered":"2024-06-02T15:40:00.024519+0000","last_clean":"2024-06-02T15:40:00.024519+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:00.024519+0000","last_undegraded":"2024-06-02T15:40:00.024519+0000","last_fullsized":"2024-06-02T15:40:00.024519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685386+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:41:27.685386+0000","last_peered":"2024-06-02T15:41:27.685386+0000","last_clean":"2024-06-02T15:41:27.685386+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:41:27.685386+0000","last_undegraded":"2024-06-02T15:41:27.685386+0000","last_fullsized":"2024-06-02T15:41:27.685386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026758+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:00.026758+0000","last_peered":"2024-06-02T15:40:00.026758+0000","last_clean":"2024-06-02T15:40:00.026758+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:00.026758+0000","last_undegraded":"2024-06-02T15:40:00.026758+0000","last_fullsized":"2024-06-02T15:40:00.026758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029906+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:00.029906+0000","last_peered":"2024-06-02T15:40:00.029906+0000","last_clean":"2024-06-02T15:40:00.029906+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:00.029906+0000","last_undegraded":"2024-06-02T15:40:00.029906+0000","last_fullsized":"2024-06-02T15:40:00.029906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032707+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:00.032707+0000","last_peered":"2024-06-02T15:40:00.032707+0000","last_clean":"2024-06-02T15:40:00.032707+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:00.032707+0000","last_undegraded":"2024-06-02T15:40:00.032707+0000","last_fullsized":"2024-06-02T15:40:00.032707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693839+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:41:27.693839+0000","last_peered":"2024-06-02T15:41:27.693839+0000","last_clean":"2024-06-02T15:41:27.693839+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:41:27.693839+0000","last_undegraded":"2024-06-02T15:41:27.693839+0000","last_fullsized":"2024-06-02T15:41:27.693839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032044+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:00.032044+0000","last_peered":"2024-06-02T15:40:00.032044+0000","last_clean":"2024-06-02T15:40:00.032044+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:00.032044+0000","last_undegraded":"2024-06-02T15:40:00.032044+0000","last_fullsized":"2024-06-02T15:40:00.032044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:25.850486+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:40:25.850486+0000","last_peered":"2024-06-02T15:40:25.850486+0000","last_clean":"2024-06-02T15:40:25.850486+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:40:25.850486+0000","last_undegraded":"2024-06-02T15:40:25.850486+0000","last_fullsized":"2024-06-02T15:40:25.850486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704498+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:41:27.704498+0000","last_peered":"2024-06-02T15:41:27.704498+0000","last_clean":"2024-06-02T15:41:27.704498+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:41:27.704498+0000","last_undegraded":"2024-06-02T15:41:27.704498+0000","last_fullsized":"2024-06-02T15:41:27.704498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030566+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:00.030566+0000","last_peered":"2024-06-02T15:40:00.030566+0000","last_clean":"2024-06-02T15:40:00.030566+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:00.030566+0000","last_undegraded":"2024-06-02T15:40:00.030566+0000","last_fullsized":"2024-06-02T15:40:00.030566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031666+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:00.031666+0000","last_peered":"2024-06-02T15:40:00.031666+0000","last_clean":"2024-06-02T15:40:00.031666+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:00.031666+0000","last_undegraded":"2024-06-02T15:40:00.031666+0000","last_fullsized":"2024-06-02T15:40:00.031666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711581+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:41:27.711581+0000","last_peered":"2024-06-02T15:41:27.711581+0000","last_clean":"2024-06-02T15:41:27.711581+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:41:27.711581+0000","last_undegraded":"2024-06-02T15:41:27.711581+0000","last_fullsized":"2024-06-02T15:41:27.711581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031524+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:00.031524+0000","last_peered":"2024-06-02T15:40:00.031524+0000","last_clean":"2024-06-02T15:40:00.031524+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:00.031524+0000","last_undegraded":"2024-06-02T15:40:00.031524+0000","last_fullsized":"2024-06-02T15:40:00.031524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027646+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:00.027646+0000","last_peered":"2024-06-02T15:40:00.027646+0000","last_clean":"2024-06-02T15:40:00.027646+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:00.027646+0000","last_undegraded":"2024-06-02T15:40:00.027646+0000","last_fullsized":"2024-06-02T15:40:00.027646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.662919+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:41:27.662919+0000","last_peered":"2024-06-02T15:41:27.662919+0000","last_clean":"2024-06-02T15:41:27.662919+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:41:27.662919+0000","last_undegraded":"2024-06-02T15:41:27.662919+0000","last_fullsized":"2024-06-02T15:41:27.662919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036643+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:00.036643+0000","last_peered":"2024-06-02T15:40:00.036643+0000","last_clean":"2024-06-02T15:40:00.036643+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:00.036643+0000","last_undegraded":"2024-06-02T15:40:00.036643+0000","last_fullsized":"2024-06-02T15:40:00.036643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032146+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:00.032146+0000","last_peered":"2024-06-02T15:40:00.032146+0000","last_clean":"2024-06-02T15:40:00.032146+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:00.032146+0000","last_undegraded":"2024-06-02T15:40:00.032146+0000","last_fullsized":"2024-06-02T15:40:00.032146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703853+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:41:27.703853+0000","last_peered":"2024-06-02T15:41:27.703853+0000","last_clean":"2024-06-02T15:41:27.703853+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:41:27.703853+0000","last_undegraded":"2024-06-02T15:41:27.703853+0000","last_fullsized":"2024-06-02T15:41:27.703853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032349+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:00.032349+0000","last_peered":"2024-06-02T15:40:00.032349+0000","last_clean":"2024-06-02T15:40:00.032349+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:00.032349+0000","last_undegraded":"2024-06-02T15:40:00.032349+0000","last_fullsized":"2024-06-02T15:40:00.032349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029769+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:40:00.029769+0000","last_peered":"2024-06-02T15:40:00.029769+0000","last_clean":"2024-06-02T15:40:00.029769+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:40:00.029769+0000","last_undegraded":"2024-06-02T15:40:00.029769+0000","last_fullsized":"2024-06-02T15:40:00.029769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711064+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:41:27.711064+0000","last_peered":"2024-06-02T15:41:27.711064+0000","last_clean":"2024-06-02T15:41:27.711064+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:41:27.711064+0000","last_undegraded":"2024-06-02T15:41:27.711064+0000","last_fullsized":"2024-06-02T15:41:27.711064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031484+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:00.031484+0000","last_peered":"2024-06-02T15:40:00.031484+0000","last_clean":"2024-06-02T15:40:00.031484+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:00.031484+0000","last_undegraded":"2024-06-02T15:40:00.031484+0000","last_fullsized":"2024-06-02T15:40:00.031484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032206+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:00.032206+0000","last_peered":"2024-06-02T15:40:00.032206+0000","last_clean":"2024-06-02T15:40:00.032206+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:00.032206+0000","last_undegraded":"2024-06-02T15:40:00.032206+0000","last_fullsized":"2024-06-02T15:40:00.032206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692699+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:41:27.692699+0000","last_peered":"2024-06-02T15:41:27.692699+0000","last_clean":"2024-06-02T15:41:27.692699+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:41:27.692699+0000","last_undegraded":"2024-06-02T15:41:27.692699+0000","last_fullsized":"2024-06-02T15:41:27.692699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027822+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:00.027822+0000","last_peered":"2024-06-02T15:40:00.027822+0000","last_clean":"2024-06-02T15:40:00.027822+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:00.027822+0000","last_undegraded":"2024-06-02T15:40:00.027822+0000","last_fullsized":"2024-06-02T15:40:00.027822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031075+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:00.031075+0000","last_peered":"2024-06-02T15:40:00.031075+0000","last_clean":"2024-06-02T15:40:00.031075+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:00.031075+0000","last_undegraded":"2024-06-02T15:40:00.031075+0000","last_fullsized":"2024-06-02T15:40:00.031075+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694351+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:41:27.694351+0000","last_peered":"2024-06-02T15:41:27.694351+0000","last_clean":"2024-06-02T15:41:27.694351+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:41:27.694351+0000","last_undegraded":"2024-06-02T15:41:27.694351+0000","last_fullsized":"2024-06-02T15:41:27.694351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035861+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:00.035861+0000","last_peered":"2024-06-02T15:40:00.035861+0000","last_clean":"2024-06-02T15:40:00.035861+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:00.035861+0000","last_undegraded":"2024-06-02T15:40:00.035861+0000","last_fullsized":"2024-06-02T15:40:00.035861+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024333+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:00.024333+0000","last_peered":"2024-06-02T15:40:00.024333+0000","last_clean":"2024-06-02T15:40:00.024333+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:00.024333+0000","last_undegraded":"2024-06-02T15:40:00.024333+0000","last_fullsized":"2024-06-02T15:40:00.024333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705089+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:41:27.705089+0000","last_peered":"2024-06-02T15:41:27.705089+0000","last_clean":"2024-06-02T15:41:27.705089+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:41:27.705089+0000","last_undegraded":"2024-06-02T15:41:27.705089+0000","last_fullsized":"2024-06-02T15:41:27.705089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023720+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:00.023720+0000","last_peered":"2024-06-02T15:40:00.023720+0000","last_clean":"2024-06-02T15:40:00.023720+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:00.023720+0000","last_undegraded":"2024-06-02T15:40:00.023720+0000","last_fullsized":"2024-06-02T15:40:00.023720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035036+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:00.035036+0000","last_peered":"2024-06-02T15:40:00.035036+0000","last_clean":"2024-06-02T15:40:00.035036+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:00.035036+0000","last_undegraded":"2024-06-02T15:40:00.035036+0000","last_fullsized":"2024-06-02T15:40:00.035036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684451+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:41:27.684451+0000","last_peered":"2024-06-02T15:41:27.684451+0000","last_clean":"2024-06-02T15:41:27.684451+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:41:27.684451+0000","last_undegraded":"2024-06-02T15:41:27.684451+0000","last_fullsized":"2024-06-02T15:41:27.684451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687108+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:41:27.687108+0000","last_peered":"2024-06-02T15:41:27.687108+0000","last_clean":"2024-06-02T15:41:27.687108+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:41:27.687108+0000","last_undegraded":"2024-06-02T15:41:27.687108+0000","last_fullsized":"2024-06-02T15:41:27.687108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686042+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:41:27.686042+0000","last_peered":"2024-06-02T15:41:27.686042+0000","last_clean":"2024-06-02T15:41:27.686042+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:41:27.686042+0000","last_undegraded":"2024-06-02T15:41:27.686042+0000","last_fullsized":"2024-06-02T15:41:27.686042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684500+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:41:27.684500+0000","last_peered":"2024-06-02T15:41:27.684500+0000","last_clean":"2024-06-02T15:41:27.684500+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:41:27.684500+0000","last_undegraded":"2024-06-02T15:41:27.684500+0000","last_fullsized":"2024-06-02T15:41:27.684500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701840+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:41:27.701840+0000","last_peered":"2024-06-02T15:41:27.701840+0000","last_clean":"2024-06-02T15:41:27.701840+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:41:27.701840+0000","last_undegraded":"2024-06-02T15:41:27.701840+0000","last_fullsized":"2024-06-02T15:41:27.701840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704303+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:41:27.704303+0000","last_peered":"2024-06-02T15:41:27.704303+0000","last_clean":"2024-06-02T15:41:27.704303+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:41:27.704303+0000","last_undegraded":"2024-06-02T15:41:27.704303+0000","last_fullsized":"2024-06-02T15:41:27.704303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684291+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:41:27.684291+0000","last_peered":"2024-06-02T15:41:27.684291+0000","last_clean":"2024-06-02T15:41:27.684291+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:41:27.684291+0000","last_undegraded":"2024-06-02T15:41:27.684291+0000","last_fullsized":"2024-06-02T15:41:27.684291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.691113+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:41:27.691113+0000","last_peered":"2024-06-02T15:41:27.691113+0000","last_clean":"2024-06-02T15:41:27.691113+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:41:27.691113+0000","last_undegraded":"2024-06-02T15:41:27.691113+0000","last_fullsized":"2024-06-02T15:41:27.691113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701815+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:41:27.701815+0000","last_peered":"2024-06-02T15:41:27.701815+0000","last_clean":"2024-06-02T15:41:27.701815+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:41:27.701815+0000","last_undegraded":"2024-06-02T15:41:27.701815+0000","last_fullsized":"2024-06-02T15:41:27.701815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711699+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:41:27.711699+0000","last_peered":"2024-06-02T15:41:27.711699+0000","last_clean":"2024-06-02T15:41:27.711699+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:41:27.711699+0000","last_undegraded":"2024-06-02T15:41:27.711699+0000","last_fullsized":"2024-06-02T15:41:27.711699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705567+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:41:27.705567+0000","last_peered":"2024-06-02T15:41:27.705567+0000","last_clean":"2024-06-02T15:41:27.705567+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:41:27.705567+0000","last_undegraded":"2024-06-02T15:41:27.705567+0000","last_fullsized":"2024-06-02T15:41:27.705567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711796+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:41:27.711796+0000","last_peered":"2024-06-02T15:41:27.711796+0000","last_clean":"2024-06-02T15:41:27.711796+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:41:27.711796+0000","last_undegraded":"2024-06-02T15:41:27.711796+0000","last_fullsized":"2024-06-02T15:41:27.711796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703813+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:41:27.703813+0000","last_peered":"2024-06-02T15:41:27.703813+0000","last_clean":"2024-06-02T15:41:27.703813+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:41:27.703813+0000","last_undegraded":"2024-06-02T15:41:27.703813+0000","last_fullsized":"2024-06-02T15:41:27.703813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698643+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:41:27.698643+0000","last_peered":"2024-06-02T15:41:27.698643+0000","last_clean":"2024-06-02T15:41:27.698643+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:41:27.698643+0000","last_undegraded":"2024-06-02T15:41:27.698643+0000","last_fullsized":"2024-06-02T15:41:27.698643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68983,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":768,"num_read_kb":458,"num_write":180,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678076,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874303,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1170269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037830,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234061,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430289,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1114316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":577791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":515364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149799,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":583063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790045,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1105858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:41:43.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T15:41:43.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:42 smithi120 ceph-mon[36451]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:41:43.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:42 smithi120 ceph-mon[36451]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:43.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:42 smithi135 ceph-mon[36839]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:41:43.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:42 smithi135 ceph-mon[36839]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:43.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:42 smithi090 ceph-mon[31595]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:41:43.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:42 smithi090 ceph-mon[31595]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:45.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:44 smithi090 ceph-mon[31595]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-02T15:41:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:44 smithi120 ceph-mon[36451]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-02T15:41:45.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:44 smithi135 ceph-mon[36839]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.703+0000 7f2e80600700 1 -- 172.21.15.90:0/3866849280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 msgr2=0x7f2e78110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.703+0000 7f2e80600700 1 --2- 172.21.15.90:0/3866849280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e78110470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009f90 tx=0x7f2e7000b330 comp rx=0 tx=0).stop 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.703+0000 7f2e80600700 1 -- 172.21.15.90:0/3866849280 shutdown_connections 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.703+0000 7f2e80600700 1 --2- 172.21.15.90:0/3866849280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e78110470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.704+0000 7f2e80600700 1 --2- 172.21.15.90:0/3866849280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.704+0000 7f2e80600700 1 --2- 172.21.15.90:0/3866849280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 0x7f2e781081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.704+0000 7f2e80600700 1 -- 172.21.15.90:0/3866849280 >> 172.21.15.90:0/3866849280 conn(0x7f2e7807a6a0 msgr2=0x7f2e7807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:45.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.704+0000 7f2e80600700 1 -- 172.21.15.90:0/3866849280 shutdown_connections 2024-06-02T15:41:45.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.704+0000 7f2e80600700 1 -- 172.21.15.90:0/3866849280 wait complete. 2024-06-02T15:41:45.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 Processor -- start 2024-06-02T15:41:45.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 -- start start 2024-06-02T15:41:45.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 0x7f2e78075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e780762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2e781136c0 con 0x7f2e781037f0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113540 con 0x7f2e781041a0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.705+0000 7f2e80600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78113840 con 0x7f2e781088f0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39818/0 (socket says 172.21.15.90:39818) 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 -- 172.21.15.90:0/1710543535 learned_addr learned my addr 172.21.15.90:0/1710543535 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7e600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e780762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7d200700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 0x7f2e78075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:45.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 -- 172.21.15.90:0/1710543535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 msgr2=0x7f2e780762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e780762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 -- 172.21.15.90:0/1710543535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 msgr2=0x7f2e78075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 0x7f2e78075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.706+0000 7f2e7dc00700 1 -- 172.21.15.90:0/1710543535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e781a8020 con 0x7f2e781037f0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e7dc00700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00eb30 tx=0x7f2e6c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00cbd0 con 0x7f2e781037f0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e781a8310 con 0x7f2e781037f0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6c0044d0 con 0x7f2e781037f0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c003e30 con 0x7f2e781037f0 2024-06-02T15:41:45.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.707+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781a8850 con 0x7f2e781037f0 2024-06-02T15:41:45.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.708+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7806a230 con 0x7f2e781037f0 2024-06-02T15:41:45.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.710+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2e6c004000 con 0x7f2e781037f0 2024-06-02T15:41:45.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.712+0000 7f2e76000700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:45.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.712+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2e6c0e29b0 con 0x7f2e781037f0 2024-06-02T15:41:45.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.712+0000 7f2e7d200700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:45.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.713+0000 7f2e7d200700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 0x7f2e600b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009750 tx=0x7f2e64009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:45.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.717+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e6c0042f0 con 0x7f2e781037f0 2024-06-02T15:41:45.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.943+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e78063980 con 0x7f2e600b2880 2024-06-02T15:41:45.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.961+0000 7f2e76000700 1 -- 172.21.15.90:0/1710543535 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340481 (secure 0 0 0) 0x7f2e78063980 con 0x7f2e600b2880 2024-06-02T15:41:45.964 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 msgr2=0x7f2e600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 0x7f2e600b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009750 tx=0x7f2e64009310 comp rx=0 tx=0).stop 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 msgr2=0x7f2e78075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00eb30 tx=0x7f2e6c00ee40 comp rx=0 tx=0).stop 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 shutdown_connections 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e781088f0 0x7f2e780762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e781037f0 0x7f2e78075840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 --2- 172.21.15.90:0/1710543535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2e781041a0 0x7f2e78075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 >> 172.21.15.90:0/1710543535 conn(0x7f2e7807a6a0 msgr2=0x7f2e78101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 shutdown_connections 2024-06-02T15:41:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:45.967+0000 7f2e80600700 1 -- 172.21.15.90:0/1710543535 wait complete. 2024-06-02T15:41:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:41:46.761 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-06-02T15:41:45.776152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659585,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":874,"num_read_kb":537,"num_write":305,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":748,"ondisk_log_size":748,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375824,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566000,"statfs":{"total":1151940427776,"available":1151555584000,"internally_reserved":0,"allocated":14532608,"data_stored":8172605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005657"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706782+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:41:27.706782+0000","last_peered":"2024-06-02T15:41:27.706782+0000","last_clean":"2024-06-02T15:41:27.706782+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:41:27.706782+0000","last_undegraded":"2024-06-02T15:41:27.706782+0000","last_fullsized":"2024-06-02T15:41:27.706782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703557+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:41:27.703557+0000","last_peered":"2024-06-02T15:41:27.703557+0000","last_clean":"2024-06-02T15:41:27.703557+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:41:27.703557+0000","last_undegraded":"2024-06-02T15:41:27.703557+0000","last_fullsized":"2024-06-02T15:41:27.703557+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706429+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:41:27.706429+0000","last_peered":"2024-06-02T15:41:27.706429+0000","last_clean":"2024-06-02T15:41:27.706429+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:41:27.706429+0000","last_undegraded":"2024-06-02T15:41:27.706429+0000","last_fullsized":"2024-06-02T15:41:27.706429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692023+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:41:27.692023+0000","last_peered":"2024-06-02T15:41:27.692023+0000","last_clean":"2024-06-02T15:41:27.692023+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:41:27.692023+0000","last_undegraded":"2024-06-02T15:41:27.692023+0000","last_fullsized":"2024-06-02T15:41:27.692023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699088+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:41:27.699088+0000","last_peered":"2024-06-02T15:41:27.699088+0000","last_clean":"2024-06-02T15:41:27.699088+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:41:27.699088+0000","last_undegraded":"2024-06-02T15:41:27.699088+0000","last_fullsized":"2024-06-02T15:41:27.699088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697398+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:41:27.697398+0000","last_peered":"2024-06-02T15:41:27.697398+0000","last_clean":"2024-06-02T15:41:27.697398+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:41:27.697398+0000","last_undegraded":"2024-06-02T15:41:27.697398+0000","last_fullsized":"2024-06-02T15:41:27.697398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.700624+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:41:27.700624+0000","last_peered":"2024-06-02T15:41:27.700624+0000","last_clean":"2024-06-02T15:41:27.700624+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:41:27.700624+0000","last_undegraded":"2024-06-02T15:41:27.700624+0000","last_fullsized":"2024-06-02T15:41:27.700624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711513+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:41:27.711513+0000","last_peered":"2024-06-02T15:41:27.711513+0000","last_clean":"2024-06-02T15:41:27.711513+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:41:27.711513+0000","last_undegraded":"2024-06-02T15:41:27.711513+0000","last_fullsized":"2024-06-02T15:41:27.711513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692843+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:41:27.692843+0000","last_peered":"2024-06-02T15:41:27.692843+0000","last_clean":"2024-06-02T15:41:27.692843+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:41:27.692843+0000","last_undegraded":"2024-06-02T15:41:27.692843+0000","last_fullsized":"2024-06-02T15:41:27.692843+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698540+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:41:27.698540+0000","last_peered":"2024-06-02T15:41:27.698540+0000","last_clean":"2024-06-02T15:41:27.698540+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:41:27.698540+0000","last_undegraded":"2024-06-02T15:41:27.698540+0000","last_fullsized":"2024-06-02T15:41:27.698540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694241+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:41:27.694241+0000","last_peered":"2024-06-02T15:41:27.694241+0000","last_clean":"2024-06-02T15:41:27.694241+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:41:27.694241+0000","last_undegraded":"2024-06-02T15:41:27.694241+0000","last_fullsized":"2024-06-02T15:41:27.694241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690004+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:41:27.690004+0000","last_peered":"2024-06-02T15:41:27.690004+0000","last_clean":"2024-06-02T15:41:27.690004+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:41:27.690004+0000","last_undegraded":"2024-06-02T15:41:27.690004+0000","last_fullsized":"2024-06-02T15:41:27.690004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694651+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:41:27.694651+0000","last_peered":"2024-06-02T15:41:27.694651+0000","last_clean":"2024-06-02T15:41:27.694651+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:41:27.694651+0000","last_undegraded":"2024-06-02T15:41:27.694651+0000","last_fullsized":"2024-06-02T15:41:27.694651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699405+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:41:27.699405+0000","last_peered":"2024-06-02T15:41:27.699405+0000","last_clean":"2024-06-02T15:41:27.699405+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:41:27.699405+0000","last_undegraded":"2024-06-02T15:41:27.699405+0000","last_fullsized":"2024-06-02T15:41:27.699405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705584+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:41:27.705584+0000","last_peered":"2024-06-02T15:41:27.705584+0000","last_clean":"2024-06-02T15:41:27.705584+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:41:27.705584+0000","last_undegraded":"2024-06-02T15:41:27.705584+0000","last_fullsized":"2024-06-02T15:41:27.705584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702871+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:41:27.702871+0000","last_peered":"2024-06-02T15:41:27.702871+0000","last_clean":"2024-06-02T15:41:27.702871+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:41:27.702871+0000","last_undegraded":"2024-06-02T15:41:27.702871+0000","last_fullsized":"2024-06-02T15:41:27.702871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706832+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:41:27.706832+0000","last_peered":"2024-06-02T15:41:27.706832+0000","last_clean":"2024-06-02T15:41:27.706832+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:41:27.706832+0000","last_undegraded":"2024-06-02T15:41:27.706832+0000","last_fullsized":"2024-06-02T15:41:27.706832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701107+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:41:27.701107+0000","last_peered":"2024-06-02T15:41:27.701107+0000","last_clean":"2024-06-02T15:41:27.701107+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:41:27.701107+0000","last_undegraded":"2024-06-02T15:41:27.701107+0000","last_fullsized":"2024-06-02T15:41:27.701107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687376+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:41:27.687376+0000","last_peered":"2024-06-02T15:41:27.687376+0000","last_clean":"2024-06-02T15:41:27.687376+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:41:27.687376+0000","last_undegraded":"2024-06-02T15:41:27.687376+0000","last_fullsized":"2024-06-02T15:41:27.687376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030556+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:00.030556+0000","last_peered":"2024-06-02T15:40:00.030556+0000","last_clean":"2024-06-02T15:40:00.030556+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:00.030556+0000","last_undegraded":"2024-06-02T15:40:00.030556+0000","last_fullsized":"2024-06-02T15:40:00.030556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":67,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036770+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:00.036770+0000","last_peered":"2024-06-02T15:40:00.036770+0000","last_clean":"2024-06-02T15:40:00.036770+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:00.036770+0000","last_undegraded":"2024-06-02T15:40:00.036770+0000","last_fullsized":"2024-06-02T15:40:00.036770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702317+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:41:27.702317+0000","last_peered":"2024-06-02T15:41:27.702317+0000","last_clean":"2024-06-02T15:41:27.702317+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:41:27.702317+0000","last_undegraded":"2024-06-02T15:41:27.702317+0000","last_fullsized":"2024-06-02T15:41:27.702317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.042403+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:00.042403+0000","last_peered":"2024-06-02T15:40:00.042403+0000","last_clean":"2024-06-02T15:40:00.042403+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:00.042403+0000","last_undegraded":"2024-06-02T15:40:00.042403+0000","last_fullsized":"2024-06-02T15:40:00.042403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034821+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:00.034821+0000","last_peered":"2024-06-02T15:40:00.034821+0000","last_clean":"2024-06-02T15:40:00.034821+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:00.034821+0000","last_undegraded":"2024-06-02T15:40:00.034821+0000","last_fullsized":"2024-06-02T15:40:00.034821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708333+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:41:27.708333+0000","last_peered":"2024-06-02T15:41:27.708333+0000","last_clean":"2024-06-02T15:41:27.708333+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:41:27.708333+0000","last_undegraded":"2024-06-02T15:41:27.708333+0000","last_fullsized":"2024-06-02T15:41:27.708333+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031438+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031438+0000","last_peered":"2024-06-02T15:40:00.031438+0000","last_clean":"2024-06-02T15:40:00.031438+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031438+0000","last_undegraded":"2024-06-02T15:40:00.031438+0000","last_fullsized":"2024-06-02T15:40:00.031438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032580+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:00.032580+0000","last_peered":"2024-06-02T15:40:00.032580+0000","last_clean":"2024-06-02T15:40:00.032580+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:00.032580+0000","last_undegraded":"2024-06-02T15:40:00.032580+0000","last_fullsized":"2024-06-02T15:40:00.032580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695456+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:41:27.695456+0000","last_peered":"2024-06-02T15:41:27.695456+0000","last_clean":"2024-06-02T15:41:27.695456+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:41:27.695456+0000","last_undegraded":"2024-06-02T15:41:27.695456+0000","last_fullsized":"2024-06-02T15:41:27.695456+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031342+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031342+0000","last_peered":"2024-06-02T15:40:00.031342+0000","last_clean":"2024-06-02T15:40:00.031342+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031342+0000","last_undegraded":"2024-06-02T15:40:00.031342+0000","last_fullsized":"2024-06-02T15:40:00.031342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032456+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:00.032456+0000","last_peered":"2024-06-02T15:40:00.032456+0000","last_clean":"2024-06-02T15:40:00.032456+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:00.032456+0000","last_undegraded":"2024-06-02T15:40:00.032456+0000","last_fullsized":"2024-06-02T15:40:00.032456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696093+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:41:27.696093+0000","last_peered":"2024-06-02T15:41:27.696093+0000","last_clean":"2024-06-02T15:41:27.696093+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:41:27.696093+0000","last_undegraded":"2024-06-02T15:41:27.696093+0000","last_fullsized":"2024-06-02T15:41:27.696093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031260+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031260+0000","last_peered":"2024-06-02T15:40:00.031260+0000","last_clean":"2024-06-02T15:40:00.031260+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031260+0000","last_undegraded":"2024-06-02T15:40:00.031260+0000","last_fullsized":"2024-06-02T15:40:00.031260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":184,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:42.329501+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:41:42.329501+0000","last_peered":"2024-06-02T15:41:42.329501+0000","last_clean":"2024-06-02T15:41:42.329501+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:41:42.329501+0000","last_undegraded":"2024-06-02T15:41:42.329501+0000","last_fullsized":"2024-06-02T15:41:42.329501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697520+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:41:27.697520+0000","last_peered":"2024-06-02T15:41:27.697520+0000","last_clean":"2024-06-02T15:41:27.697520+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:41:27.697520+0000","last_undegraded":"2024-06-02T15:41:27.697520+0000","last_fullsized":"2024-06-02T15:41:27.697520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031151+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031151+0000","last_peered":"2024-06-02T15:40:00.031151+0000","last_clean":"2024-06-02T15:40:00.031151+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031151+0000","last_undegraded":"2024-06-02T15:40:00.031151+0000","last_fullsized":"2024-06-02T15:40:00.031151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031630+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:00.031630+0000","last_peered":"2024-06-02T15:40:00.031630+0000","last_clean":"2024-06-02T15:40:00.031630+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:00.031630+0000","last_undegraded":"2024-06-02T15:40:00.031630+0000","last_fullsized":"2024-06-02T15:40:00.031630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711290+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:41:27.711290+0000","last_peered":"2024-06-02T15:41:27.711290+0000","last_clean":"2024-06-02T15:41:27.711290+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:41:27.711290+0000","last_undegraded":"2024-06-02T15:41:27.711290+0000","last_fullsized":"2024-06-02T15:41:27.711290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031144+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:00.031144+0000","last_peered":"2024-06-02T15:40:00.031144+0000","last_clean":"2024-06-02T15:40:00.031144+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:00.031144+0000","last_undegraded":"2024-06-02T15:40:00.031144+0000","last_fullsized":"2024-06-02T15:40:00.031144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.033997+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:00.033997+0000","last_peered":"2024-06-02T15:40:00.033997+0000","last_clean":"2024-06-02T15:40:00.033997+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:00.033997+0000","last_undegraded":"2024-06-02T15:40:00.033997+0000","last_fullsized":"2024-06-02T15:40:00.033997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711334+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:41:27.711334+0000","last_peered":"2024-06-02T15:41:27.711334+0000","last_clean":"2024-06-02T15:41:27.711334+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:41:27.711334+0000","last_undegraded":"2024-06-02T15:41:27.711334+0000","last_fullsized":"2024-06-02T15:41:27.711334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026122+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:00.026122+0000","last_peered":"2024-06-02T15:40:00.026122+0000","last_clean":"2024-06-02T15:40:00.026122+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:00.026122+0000","last_undegraded":"2024-06-02T15:40:00.026122+0000","last_fullsized":"2024-06-02T15:40:00.026122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023805+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:00.023805+0000","last_peered":"2024-06-02T15:40:00.023805+0000","last_clean":"2024-06-02T15:40:00.023805+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:00.023805+0000","last_undegraded":"2024-06-02T15:40:00.023805+0000","last_fullsized":"2024-06-02T15:40:00.023805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710271+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:41:27.710271+0000","last_peered":"2024-06-02T15:41:27.710271+0000","last_clean":"2024-06-02T15:41:27.710271+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:41:27.710271+0000","last_undegraded":"2024-06-02T15:41:27.710271+0000","last_fullsized":"2024-06-02T15:41:27.710271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032043+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:00.032043+0000","last_peered":"2024-06-02T15:40:00.032043+0000","last_clean":"2024-06-02T15:40:00.032043+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:00.032043+0000","last_undegraded":"2024-06-02T15:40:00.032043+0000","last_fullsized":"2024-06-02T15:40:00.032043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:41.327637+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:41:41.327637+0000","last_peered":"2024-06-02T15:41:41.327637+0000","last_clean":"2024-06-02T15:41:41.327637+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:41:41.327637+0000","last_undegraded":"2024-06-02T15:41:41.327637+0000","last_fullsized":"2024-06-02T15:41:41.327637+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701017+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:41:27.701017+0000","last_peered":"2024-06-02T15:41:27.701017+0000","last_clean":"2024-06-02T15:41:27.701017+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:41:27.701017+0000","last_undegraded":"2024-06-02T15:41:27.701017+0000","last_fullsized":"2024-06-02T15:41:27.701017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032926+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:00.032926+0000","last_peered":"2024-06-02T15:40:00.032926+0000","last_clean":"2024-06-02T15:40:00.032926+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:00.032926+0000","last_undegraded":"2024-06-02T15:40:00.032926+0000","last_fullsized":"2024-06-02T15:40:00.032926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027472+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:00.027472+0000","last_peered":"2024-06-02T15:40:00.027472+0000","last_clean":"2024-06-02T15:40:00.027472+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:00.027472+0000","last_undegraded":"2024-06-02T15:40:00.027472+0000","last_fullsized":"2024-06-02T15:40:00.027472+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706776+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:41:27.706776+0000","last_peered":"2024-06-02T15:41:27.706776+0000","last_clean":"2024-06-02T15:41:27.706776+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:41:27.706776+0000","last_undegraded":"2024-06-02T15:41:27.706776+0000","last_fullsized":"2024-06-02T15:41:27.706776+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031950+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:00.031950+0000","last_peered":"2024-06-02T15:40:00.031950+0000","last_clean":"2024-06-02T15:40:00.031950+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:00.031950+0000","last_undegraded":"2024-06-02T15:40:00.031950+0000","last_fullsized":"2024-06-02T15:40:00.031950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024224+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:00.024224+0000","last_peered":"2024-06-02T15:40:00.024224+0000","last_clean":"2024-06-02T15:40:00.024224+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:00.024224+0000","last_undegraded":"2024-06-02T15:40:00.024224+0000","last_fullsized":"2024-06-02T15:40:00.024224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698155+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:41:27.698155+0000","last_peered":"2024-06-02T15:41:27.698155+0000","last_clean":"2024-06-02T15:41:27.698155+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:41:27.698155+0000","last_undegraded":"2024-06-02T15:41:27.698155+0000","last_fullsized":"2024-06-02T15:41:27.698155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034406+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:00.034406+0000","last_peered":"2024-06-02T15:40:00.034406+0000","last_clean":"2024-06-02T15:40:00.034406+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:00.034406+0000","last_undegraded":"2024-06-02T15:40:00.034406+0000","last_fullsized":"2024-06-02T15:40:00.034406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":362,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:42.328859+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:41:42.328859+0000","last_peered":"2024-06-02T15:41:42.328859+0000","last_clean":"2024-06-02T15:41:42.328859+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:41:42.328859+0000","last_undegraded":"2024-06-02T15:41:42.328859+0000","last_fullsized":"2024-06-02T15:41:42.328859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64309,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":264,"num_read_kb":195,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685689+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:41:27.685689+0000","last_peered":"2024-06-02T15:41:27.685689+0000","last_clean":"2024-06-02T15:41:27.685689+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:41:27.685689+0000","last_undegraded":"2024-06-02T15:41:27.685689+0000","last_fullsized":"2024-06-02T15:41:27.685689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024687+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:00.024687+0000","last_peered":"2024-06-02T15:40:00.024687+0000","last_clean":"2024-06-02T15:40:00.024687+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:00.024687+0000","last_undegraded":"2024-06-02T15:40:00.024687+0000","last_fullsized":"2024-06-02T15:40:00.024687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030857+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:00.030857+0000","last_peered":"2024-06-02T15:40:00.030857+0000","last_clean":"2024-06-02T15:40:00.030857+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:00.030857+0000","last_undegraded":"2024-06-02T15:40:00.030857+0000","last_fullsized":"2024-06-02T15:40:00.030857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710858+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:41:27.710858+0000","last_peered":"2024-06-02T15:41:27.710858+0000","last_clean":"2024-06-02T15:41:27.710858+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:41:27.710858+0000","last_undegraded":"2024-06-02T15:41:27.710858+0000","last_fullsized":"2024-06-02T15:41:27.710858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030018+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:00.030018+0000","last_peered":"2024-06-02T15:40:00.030018+0000","last_clean":"2024-06-02T15:40:00.030018+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:00.030018+0000","last_undegraded":"2024-06-02T15:40:00.030018+0000","last_fullsized":"2024-06-02T15:40:00.030018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024214+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:00.024214+0000","last_peered":"2024-06-02T15:40:00.024214+0000","last_clean":"2024-06-02T15:40:00.024214+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:00.024214+0000","last_undegraded":"2024-06-02T15:40:00.024214+0000","last_fullsized":"2024-06-02T15:40:00.024214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696457+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:41:27.696457+0000","last_peered":"2024-06-02T15:41:27.696457+0000","last_clean":"2024-06-02T15:41:27.696457+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:41:27.696457+0000","last_undegraded":"2024-06-02T15:41:27.696457+0000","last_fullsized":"2024-06-02T15:41:27.696457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036506+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:00.036506+0000","last_peered":"2024-06-02T15:40:00.036506+0000","last_clean":"2024-06-02T15:40:00.036506+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:00.036506+0000","last_undegraded":"2024-06-02T15:40:00.036506+0000","last_fullsized":"2024-06-02T15:40:00.036506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":182,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:42.398493+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:41:42.398493+0000","last_peered":"2024-06-02T15:41:42.398493+0000","last_clean":"2024-06-02T15:41:42.398493+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:41:42.398493+0000","last_undegraded":"2024-06-02T15:41:42.398493+0000","last_fullsized":"2024-06-02T15:41:42.398493+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687441+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:41:27.687441+0000","last_peered":"2024-06-02T15:41:27.687441+0000","last_clean":"2024-06-02T15:41:27.687441+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:41:27.687441+0000","last_undegraded":"2024-06-02T15:41:27.687441+0000","last_fullsized":"2024-06-02T15:41:27.687441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.038360+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:00.038360+0000","last_peered":"2024-06-02T15:40:00.038360+0000","last_clean":"2024-06-02T15:40:00.038360+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:00.038360+0000","last_undegraded":"2024-06-02T15:40:00.038360+0000","last_fullsized":"2024-06-02T15:40:00.038360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032025+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:00.032025+0000","last_peered":"2024-06-02T15:40:00.032025+0000","last_clean":"2024-06-02T15:40:00.032025+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:00.032025+0000","last_undegraded":"2024-06-02T15:40:00.032025+0000","last_fullsized":"2024-06-02T15:40:00.032025+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694331+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:41:27.694331+0000","last_peered":"2024-06-02T15:41:27.694331+0000","last_clean":"2024-06-02T15:41:27.694331+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:41:27.694331+0000","last_undegraded":"2024-06-02T15:41:27.694331+0000","last_fullsized":"2024-06-02T15:41:27.694331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024046+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:00.024046+0000","last_peered":"2024-06-02T15:40:00.024046+0000","last_clean":"2024-06-02T15:40:00.024046+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:00.024046+0000","last_undegraded":"2024-06-02T15:40:00.024046+0000","last_fullsized":"2024-06-02T15:40:00.024046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032813+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:00.032813+0000","last_peered":"2024-06-02T15:40:00.032813+0000","last_clean":"2024-06-02T15:40:00.032813+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:00.032813+0000","last_undegraded":"2024-06-02T15:40:00.032813+0000","last_fullsized":"2024-06-02T15:40:00.032813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695201+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:41:27.695201+0000","last_peered":"2024-06-02T15:41:27.695201+0000","last_clean":"2024-06-02T15:41:27.695201+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:41:27.695201+0000","last_undegraded":"2024-06-02T15:41:27.695201+0000","last_fullsized":"2024-06-02T15:41:27.695201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032119+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:00.032119+0000","last_peered":"2024-06-02T15:40:00.032119+0000","last_clean":"2024-06-02T15:40:00.032119+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:00.032119+0000","last_undegraded":"2024-06-02T15:40:00.032119+0000","last_fullsized":"2024-06-02T15:40:00.032119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035913+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:00.035913+0000","last_peered":"2024-06-02T15:40:00.035913+0000","last_clean":"2024-06-02T15:40:00.035913+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:00.035913+0000","last_undegraded":"2024-06-02T15:40:00.035913+0000","last_fullsized":"2024-06-02T15:40:00.035913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693160+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:41:27.693160+0000","last_peered":"2024-06-02T15:41:27.693160+0000","last_clean":"2024-06-02T15:41:27.693160+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:41:27.693160+0000","last_undegraded":"2024-06-02T15:41:27.693160+0000","last_fullsized":"2024-06-02T15:41:27.693160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.034575+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:00.034575+0000","last_peered":"2024-06-02T15:40:00.034575+0000","last_clean":"2024-06-02T15:40:00.034575+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:00.034575+0000","last_undegraded":"2024-06-02T15:40:00.034575+0000","last_fullsized":"2024-06-02T15:40:00.034575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026934+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:00.026934+0000","last_peered":"2024-06-02T15:40:00.026934+0000","last_clean":"2024-06-02T15:40:00.026934+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:00.026934+0000","last_undegraded":"2024-06-02T15:40:00.026934+0000","last_fullsized":"2024-06-02T15:40:00.026934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.676816+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:41:27.676816+0000","last_peered":"2024-06-02T15:41:27.676816+0000","last_clean":"2024-06-02T15:41:27.676816+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:41:27.676816+0000","last_undegraded":"2024-06-02T15:41:27.676816+0000","last_fullsized":"2024-06-02T15:41:27.676816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.042186+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:00.042186+0000","last_peered":"2024-06-02T15:40:00.042186+0000","last_clean":"2024-06-02T15:40:00.042186+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:00.042186+0000","last_undegraded":"2024-06-02T15:40:00.042186+0000","last_fullsized":"2024-06-02T15:40:00.042186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686863+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:41:27.686863+0000","last_peered":"2024-06-02T15:41:27.686863+0000","last_clean":"2024-06-02T15:41:27.686863+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:41:27.686863+0000","last_undegraded":"2024-06-02T15:41:27.686863+0000","last_fullsized":"2024-06-02T15:41:27.686863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:43.399679+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:41:43.399679+0000","last_peered":"2024-06-02T15:41:43.399679+0000","last_clean":"2024-06-02T15:41:43.399679+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:41:43.399679+0000","last_undegraded":"2024-06-02T15:41:43.399679+0000","last_fullsized":"2024-06-02T15:41:43.399679+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.045737+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:00.045737+0000","last_peered":"2024-06-02T15:40:00.045737+0000","last_clean":"2024-06-02T15:40:00.045737+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:00.045737+0000","last_undegraded":"2024-06-02T15:40:00.045737+0000","last_fullsized":"2024-06-02T15:40:00.045737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703766+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:41:27.703766+0000","last_peered":"2024-06-02T15:41:27.703766+0000","last_clean":"2024-06-02T15:41:27.703766+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:41:27.703766+0000","last_undegraded":"2024-06-02T15:41:27.703766+0000","last_fullsized":"2024-06-02T15:41:27.703766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031825+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:00.031825+0000","last_peered":"2024-06-02T15:40:00.031825+0000","last_clean":"2024-06-02T15:40:00.031825+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:00.031825+0000","last_undegraded":"2024-06-02T15:40:00.031825+0000","last_fullsized":"2024-06-02T15:40:00.031825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026616+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:40:00.026616+0000","last_peered":"2024-06-02T15:40:00.026616+0000","last_clean":"2024-06-02T15:40:00.026616+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:40:00.026616+0000","last_undegraded":"2024-06-02T15:40:00.026616+0000","last_fullsized":"2024-06-02T15:40:00.026616+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024519+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:00.024519+0000","last_peered":"2024-06-02T15:40:00.024519+0000","last_clean":"2024-06-02T15:40:00.024519+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:00.024519+0000","last_undegraded":"2024-06-02T15:40:00.024519+0000","last_fullsized":"2024-06-02T15:40:00.024519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685386+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:41:27.685386+0000","last_peered":"2024-06-02T15:41:27.685386+0000","last_clean":"2024-06-02T15:41:27.685386+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:41:27.685386+0000","last_undegraded":"2024-06-02T15:41:27.685386+0000","last_fullsized":"2024-06-02T15:41:27.685386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.026758+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:00.026758+0000","last_peered":"2024-06-02T15:40:00.026758+0000","last_clean":"2024-06-02T15:40:00.026758+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:00.026758+0000","last_undegraded":"2024-06-02T15:40:00.026758+0000","last_fullsized":"2024-06-02T15:40:00.026758+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029906+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:00.029906+0000","last_peered":"2024-06-02T15:40:00.029906+0000","last_clean":"2024-06-02T15:40:00.029906+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:00.029906+0000","last_undegraded":"2024-06-02T15:40:00.029906+0000","last_fullsized":"2024-06-02T15:40:00.029906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032707+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:00.032707+0000","last_peered":"2024-06-02T15:40:00.032707+0000","last_clean":"2024-06-02T15:40:00.032707+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:00.032707+0000","last_undegraded":"2024-06-02T15:40:00.032707+0000","last_fullsized":"2024-06-02T15:40:00.032707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693839+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:41:27.693839+0000","last_peered":"2024-06-02T15:41:27.693839+0000","last_clean":"2024-06-02T15:41:27.693839+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:41:27.693839+0000","last_undegraded":"2024-06-02T15:41:27.693839+0000","last_fullsized":"2024-06-02T15:41:27.693839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032044+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:00.032044+0000","last_peered":"2024-06-02T15:40:00.032044+0000","last_clean":"2024-06-02T15:40:00.032044+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:00.032044+0000","last_undegraded":"2024-06-02T15:40:00.032044+0000","last_fullsized":"2024-06-02T15:40:00.032044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'13","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:25.850486+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:40:25.850486+0000","last_peered":"2024-06-02T15:40:25.850486+0000","last_clean":"2024-06-02T15:40:25.850486+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:40:25.850486+0000","last_undegraded":"2024-06-02T15:40:25.850486+0000","last_fullsized":"2024-06-02T15:40:25.850486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704498+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:41:27.704498+0000","last_peered":"2024-06-02T15:41:27.704498+0000","last_clean":"2024-06-02T15:41:27.704498+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:41:27.704498+0000","last_undegraded":"2024-06-02T15:41:27.704498+0000","last_fullsized":"2024-06-02T15:41:27.704498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.030566+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:00.030566+0000","last_peered":"2024-06-02T15:40:00.030566+0000","last_clean":"2024-06-02T15:40:00.030566+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:00.030566+0000","last_undegraded":"2024-06-02T15:40:00.030566+0000","last_fullsized":"2024-06-02T15:40:00.030566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031666+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:00.031666+0000","last_peered":"2024-06-02T15:40:00.031666+0000","last_clean":"2024-06-02T15:40:00.031666+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:00.031666+0000","last_undegraded":"2024-06-02T15:40:00.031666+0000","last_fullsized":"2024-06-02T15:40:00.031666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711581+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:41:27.711581+0000","last_peered":"2024-06-02T15:41:27.711581+0000","last_clean":"2024-06-02T15:41:27.711581+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:41:27.711581+0000","last_undegraded":"2024-06-02T15:41:27.711581+0000","last_fullsized":"2024-06-02T15:41:27.711581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031524+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:00.031524+0000","last_peered":"2024-06-02T15:40:00.031524+0000","last_clean":"2024-06-02T15:40:00.031524+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:00.031524+0000","last_undegraded":"2024-06-02T15:40:00.031524+0000","last_fullsized":"2024-06-02T15:40:00.031524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027646+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:00.027646+0000","last_peered":"2024-06-02T15:40:00.027646+0000","last_clean":"2024-06-02T15:40:00.027646+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:00.027646+0000","last_undegraded":"2024-06-02T15:40:00.027646+0000","last_fullsized":"2024-06-02T15:40:00.027646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.662919+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:41:27.662919+0000","last_peered":"2024-06-02T15:41:27.662919+0000","last_clean":"2024-06-02T15:41:27.662919+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:41:27.662919+0000","last_undegraded":"2024-06-02T15:41:27.662919+0000","last_fullsized":"2024-06-02T15:41:27.662919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.036643+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:00.036643+0000","last_peered":"2024-06-02T15:40:00.036643+0000","last_clean":"2024-06-02T15:40:00.036643+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:00.036643+0000","last_undegraded":"2024-06-02T15:40:00.036643+0000","last_fullsized":"2024-06-02T15:40:00.036643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032146+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:00.032146+0000","last_peered":"2024-06-02T15:40:00.032146+0000","last_clean":"2024-06-02T15:40:00.032146+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:00.032146+0000","last_undegraded":"2024-06-02T15:40:00.032146+0000","last_fullsized":"2024-06-02T15:40:00.032146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703853+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:41:27.703853+0000","last_peered":"2024-06-02T15:41:27.703853+0000","last_clean":"2024-06-02T15:41:27.703853+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:41:27.703853+0000","last_undegraded":"2024-06-02T15:41:27.703853+0000","last_fullsized":"2024-06-02T15:41:27.703853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032349+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:00.032349+0000","last_peered":"2024-06-02T15:40:00.032349+0000","last_clean":"2024-06-02T15:40:00.032349+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:00.032349+0000","last_undegraded":"2024-06-02T15:40:00.032349+0000","last_fullsized":"2024-06-02T15:40:00.032349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.029769+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:40:00.029769+0000","last_peered":"2024-06-02T15:40:00.029769+0000","last_clean":"2024-06-02T15:40:00.029769+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:40:00.029769+0000","last_undegraded":"2024-06-02T15:40:00.029769+0000","last_fullsized":"2024-06-02T15:40:00.029769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711064+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:41:27.711064+0000","last_peered":"2024-06-02T15:41:27.711064+0000","last_clean":"2024-06-02T15:41:27.711064+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:41:27.711064+0000","last_undegraded":"2024-06-02T15:41:27.711064+0000","last_fullsized":"2024-06-02T15:41:27.711064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031484+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:00.031484+0000","last_peered":"2024-06-02T15:40:00.031484+0000","last_clean":"2024-06-02T15:40:00.031484+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:00.031484+0000","last_undegraded":"2024-06-02T15:40:00.031484+0000","last_fullsized":"2024-06-02T15:40:00.031484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.032206+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:00.032206+0000","last_peered":"2024-06-02T15:40:00.032206+0000","last_clean":"2024-06-02T15:40:00.032206+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:00.032206+0000","last_undegraded":"2024-06-02T15:40:00.032206+0000","last_fullsized":"2024-06-02T15:40:00.032206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692699+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:41:27.692699+0000","last_peered":"2024-06-02T15:41:27.692699+0000","last_clean":"2024-06-02T15:41:27.692699+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:41:27.692699+0000","last_undegraded":"2024-06-02T15:41:27.692699+0000","last_fullsized":"2024-06-02T15:41:27.692699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.027822+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:00.027822+0000","last_peered":"2024-06-02T15:40:00.027822+0000","last_clean":"2024-06-02T15:40:00.027822+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:00.027822+0000","last_undegraded":"2024-06-02T15:40:00.027822+0000","last_fullsized":"2024-06-02T15:40:00.027822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031075+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:00.031075+0000","last_peered":"2024-06-02T15:40:00.031075+0000","last_clean":"2024-06-02T15:40:00.031075+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:00.031075+0000","last_undegraded":"2024-06-02T15:40:00.031075+0000","last_fullsized":"2024-06-02T15:40:00.031075+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694351+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:41:27.694351+0000","last_peered":"2024-06-02T15:41:27.694351+0000","last_clean":"2024-06-02T15:41:27.694351+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:41:27.694351+0000","last_undegraded":"2024-06-02T15:41:27.694351+0000","last_fullsized":"2024-06-02T15:41:27.694351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035861+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:00.035861+0000","last_peered":"2024-06-02T15:40:00.035861+0000","last_clean":"2024-06-02T15:40:00.035861+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:00.035861+0000","last_undegraded":"2024-06-02T15:40:00.035861+0000","last_fullsized":"2024-06-02T15:40:00.035861+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.024333+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:00.024333+0000","last_peered":"2024-06-02T15:40:00.024333+0000","last_clean":"2024-06-02T15:40:00.024333+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:00.024333+0000","last_undegraded":"2024-06-02T15:40:00.024333+0000","last_fullsized":"2024-06-02T15:40:00.024333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705089+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:41:27.705089+0000","last_peered":"2024-06-02T15:41:27.705089+0000","last_clean":"2024-06-02T15:41:27.705089+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:41:27.705089+0000","last_undegraded":"2024-06-02T15:41:27.705089+0000","last_fullsized":"2024-06-02T15:41:27.705089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.023720+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:00.023720+0000","last_peered":"2024-06-02T15:40:00.023720+0000","last_clean":"2024-06-02T15:40:00.023720+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:00.023720+0000","last_undegraded":"2024-06-02T15:40:00.023720+0000","last_fullsized":"2024-06-02T15:40:00.023720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.035036+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:00.035036+0000","last_peered":"2024-06-02T15:40:00.035036+0000","last_clean":"2024-06-02T15:40:00.035036+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:00.035036+0000","last_undegraded":"2024-06-02T15:40:00.035036+0000","last_fullsized":"2024-06-02T15:40:00.035036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684451+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:41:27.684451+0000","last_peered":"2024-06-02T15:41:27.684451+0000","last_clean":"2024-06-02T15:41:27.684451+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:41:27.684451+0000","last_undegraded":"2024-06-02T15:41:27.684451+0000","last_fullsized":"2024-06-02T15:41:27.684451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687108+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:41:27.687108+0000","last_peered":"2024-06-02T15:41:27.687108+0000","last_clean":"2024-06-02T15:41:27.687108+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:41:27.687108+0000","last_undegraded":"2024-06-02T15:41:27.687108+0000","last_fullsized":"2024-06-02T15:41:27.687108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686042+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:41:27.686042+0000","last_peered":"2024-06-02T15:41:27.686042+0000","last_clean":"2024-06-02T15:41:27.686042+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:41:27.686042+0000","last_undegraded":"2024-06-02T15:41:27.686042+0000","last_fullsized":"2024-06-02T15:41:27.686042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684500+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:41:27.684500+0000","last_peered":"2024-06-02T15:41:27.684500+0000","last_clean":"2024-06-02T15:41:27.684500+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:41:27.684500+0000","last_undegraded":"2024-06-02T15:41:27.684500+0000","last_fullsized":"2024-06-02T15:41:27.684500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701840+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:41:27.701840+0000","last_peered":"2024-06-02T15:41:27.701840+0000","last_clean":"2024-06-02T15:41:27.701840+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:41:27.701840+0000","last_undegraded":"2024-06-02T15:41:27.701840+0000","last_fullsized":"2024-06-02T15:41:27.701840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704303+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:41:27.704303+0000","last_peered":"2024-06-02T15:41:27.704303+0000","last_clean":"2024-06-02T15:41:27.704303+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:41:27.704303+0000","last_undegraded":"2024-06-02T15:41:27.704303+0000","last_fullsized":"2024-06-02T15:41:27.704303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.684291+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:41:27.684291+0000","last_peered":"2024-06-02T15:41:27.684291+0000","last_clean":"2024-06-02T15:41:27.684291+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:41:27.684291+0000","last_undegraded":"2024-06-02T15:41:27.684291+0000","last_fullsized":"2024-06-02T15:41:27.684291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.691113+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:41:27.691113+0000","last_peered":"2024-06-02T15:41:27.691113+0000","last_clean":"2024-06-02T15:41:27.691113+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:41:27.691113+0000","last_undegraded":"2024-06-02T15:41:27.691113+0000","last_fullsized":"2024-06-02T15:41:27.691113+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701815+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:41:27.701815+0000","last_peered":"2024-06-02T15:41:27.701815+0000","last_clean":"2024-06-02T15:41:27.701815+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:41:27.701815+0000","last_undegraded":"2024-06-02T15:41:27.701815+0000","last_fullsized":"2024-06-02T15:41:27.701815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711699+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:41:27.711699+0000","last_peered":"2024-06-02T15:41:27.711699+0000","last_clean":"2024-06-02T15:41:27.711699+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:41:27.711699+0000","last_undegraded":"2024-06-02T15:41:27.711699+0000","last_fullsized":"2024-06-02T15:41:27.711699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705567+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:41:27.705567+0000","last_peered":"2024-06-02T15:41:27.705567+0000","last_clean":"2024-06-02T15:41:27.705567+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:41:27.705567+0000","last_undegraded":"2024-06-02T15:41:27.705567+0000","last_fullsized":"2024-06-02T15:41:27.705567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711796+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:41:27.711796+0000","last_peered":"2024-06-02T15:41:27.711796+0000","last_clean":"2024-06-02T15:41:27.711796+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:41:27.711796+0000","last_undegraded":"2024-06-02T15:41:27.711796+0000","last_fullsized":"2024-06-02T15:41:27.711796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703813+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:41:27.703813+0000","last_peered":"2024-06-02T15:41:27.703813+0000","last_clean":"2024-06-02T15:41:27.703813+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:41:27.703813+0000","last_undegraded":"2024-06-02T15:41:27.703813+0000","last_fullsized":"2024-06-02T15:41:27.703813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698643+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:41:27.698643+0000","last_peered":"2024-06-02T15:41:27.698643+0000","last_clean":"2024-06-02T15:41:27.698643+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:41:27.698643+0000","last_undegraded":"2024-06-02T15:41:27.698643+0000","last_fullsized":"2024-06-02T15:41:27.698643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68983,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":798,"num_read_kb":473,"num_write":180,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678077,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874304,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1170269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037830,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234061,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430290,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1114316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593816,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":577791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":515364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953572,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":583063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790046,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1105858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:41:46.766 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-02T15:41:46.766 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-02T15:41:46.766 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-02T15:41:46.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-02T15:41:47.250 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:47.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:46 smithi120 ceph-mon[36451]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:47.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:46 smithi120 ceph-mon[36451]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:46 smithi135 ceph-mon[36839]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:46 smithi135 ceph-mon[36839]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:46 smithi090 ceph-mon[31595]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:46 smithi090 ceph-mon[31595]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:41:49.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:48 smithi090 ceph-mon[31595]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:49.308 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:48 smithi120 ceph-mon[36451]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:49.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:48 smithi135 ceph-mon[36839]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:49.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 -- 172.21.15.90:0/1230367084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 msgr2=0x7f8d70109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 --2- 172.21.15.90:0/1230367084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d70109990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009990 tx=0x7f8d64009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 -- 172.21.15.90:0/1230367084 shutdown_connections 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 --2- 172.21.15.90:0/1230367084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006bbe0 0x7f8d701103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 --2- 172.21.15.90:0/1230367084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d70109990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 --2- 172.21.15.90:0/1230367084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006b220 0x7f8d7006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 -- 172.21.15.90:0/1230367084 >> 172.21.15.90:0/1230367084 conn(0x7f8d700793f0 msgr2=0x7f8d70079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 -- 172.21.15.90:0/1230367084 shutdown_connections 2024-06-02T15:41:49.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.477+0000 7f8d78200700 1 -- 172.21.15.90:0/1230367084 wait complete. 2024-06-02T15:41:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.478+0000 7f8d78200700 1 Processor -- start 2024-06-02T15:41:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.478+0000 7f8d78200700 1 -- start start 2024-06-02T15:41:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006bbe0 0x7f8d7019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d701a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701136e0 con 0x7f8d701095b0 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70113560 con 0x7f8d7006bbe0 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d78200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70113860 con 0x7f8d7006b220 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d75800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d76200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d701a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d75800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52600/0 (socket says 172.21.15.90:52600) 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d75800700 1 -- 172.21.15.90:0/1411256565 learned_addr learned my addr 172.21.15.90:0/1411256565 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.479+0000 7f8d74e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006bbe0 0x7f8d7019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 -- 172.21.15.90:0/1411256565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 msgr2=0x7f8d7019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 -- 172.21.15.90:0/1411256565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006bbe0 msgr2=0x7f8d7019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006bbe0 0x7f8d7019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 -- 172.21.15.90:0/1411256565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d701a0db0 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d75800700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d76200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d701a0670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007f10 tx=0x7f8d6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004db0 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68009e20 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68015400 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.480+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d701a10a0 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.481+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d701a8920 con 0x7f8d701095b0 2024-06-02T15:41:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.482+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7006a230 con 0x7f8d701095b0 2024-06-02T15:41:49.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.484+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8d68005260 con 0x7f8d701095b0 2024-06-02T15:41:49.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.485+0000 7f8d6e000700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 0x7f8d580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:49.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.486+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8d680e1ea0 con 0x7f8d701095b0 2024-06-02T15:41:49.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.486+0000 7f8d75800700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 0x7f8d580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:49.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.486+0000 7f8d75800700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 0x7f8d580b4dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009750 tx=0x7f8d6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.490+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d68010070 con 0x7f8d701095b0 2024-06-02T15:41:49.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.765+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8d701a8b50 con 0x7f8d701095b0 2024-06-02T15:41:49.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.765+0000 7f8d6e000700 1 -- 172.21.15.90:0/1411256565 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8d680a97b0 con 0x7f8d701095b0 2024-06-02T15:41:49.763 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:41:49.763 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-02T15:41:49.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.768+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 msgr2=0x7f8d580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.768+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 0x7f8d580b4dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009750 tx=0x7f8d6401a040 comp rx=0 tx=0).stop 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.768+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 msgr2=0x7f8d701a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.768+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d701a0670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007f10 tx=0x7f8d6800d3b0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 shutdown_connections 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8d580b2900 0x7f8d580b4dc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d701095b0 0x7f8d701a0670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d7006b220 0x7f8d7019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 --2- 172.21.15.90:0/1411256565 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d7006bbe0 0x7f8d7019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:49.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 >> 172.21.15.90:0/1411256565 conn(0x7f8d700793f0 msgr2=0x7f8d70100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:49.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 shutdown_connections 2024-06-02T15:41:49.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:49.769+0000 7f8d78200700 1 -- 172.21.15.90:0/1411256565 wait complete. 2024-06-02T15:41:50.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:49 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1411256565' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T15:41:50.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:49 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1411256565' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T15:41:50.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:49 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1411256565' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T15:41:50.589 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-02T15:41:50.590 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:41:51.031 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:51.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:50 smithi090 ceph-mon[31595]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:51.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:50 smithi120 ceph-mon[36451]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:51.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:50 smithi135 ceph-mon[36839]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:53.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:52 smithi090 ceph-mon[31595]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 -- 172.21.15.90:0/4035229502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 msgr2=0x7f60d0109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 --2- 172.21.15.90:0/4035229502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d0109960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009920 tx=0x7f60c0009c30 comp rx=0 tx=0).stop 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 -- 172.21.15.90:0/4035229502 shutdown_connections 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 --2- 172.21.15.90:0/4035229502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006c470 0x7f60d0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 --2- 172.21.15.90:0/4035229502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d0109960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 --2- 172.21.15.90:0/4035229502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006bab0 0x7f60d006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 -- 172.21.15.90:0/4035229502 >> 172.21.15.90:0/4035229502 conn(0x7f60d0079280 msgr2=0x7f60d0079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 -- 172.21.15.90:0/4035229502 shutdown_connections 2024-06-02T15:41:53.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.217+0000 7f60d5000700 1 -- 172.21.15.90:0/4035229502 wait complete. 2024-06-02T15:41:53.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.218+0000 7f60d5000700 1 Processor -- start 2024-06-02T15:41:53.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 -- start start 2024-06-02T15:41:53.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006bab0 0x7f60d019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:53.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:53.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:53.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113650 con 0x7f60d006c470 2024-06-02T15:41:53.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01134d0 con 0x7f60d0109580 2024-06-02T15:41:53.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.219+0000 7f60d5000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60d01137d0 con 0x7f60d006bab0 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46108/0 (socket says 172.21.15.90:46108) 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cea00700 1 -- 172.21.15.90:0/1932581360 learned_addr learned my addr 172.21.15.90:0/1932581360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cf400700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006bab0 0x7f60d019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:53.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 -- 172.21.15.90:0/1932581360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006bab0 msgr2=0x7f60d019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006bab0 0x7f60d019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 -- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 msgr2=0x7f60d019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 -- 172.21.15.90:0/1932581360 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01a0ed0 con 0x7f60d0109580 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cea00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.220+0000 7f60cfe00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d01a0790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60c800efc0 tx=0x7f60c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.221+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80093a0 con 0x7f60d0109580 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.221+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c800c940 con 0x7f60d0109580 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.221+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01a1160 con 0x7f60d0109580 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.221+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004420 con 0x7f60d0109580 2024-06-02T15:41:53.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.221+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a8a40 con 0x7f60d0109580 2024-06-02T15:41:53.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.222+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d006a230 con 0x7f60d0109580 2024-06-02T15:41:53.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.224+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f60c801b030 con 0x7f60d0109580 2024-06-02T15:41:53.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.225+0000 7f60c7e00700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 0x7f60b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:53.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.226+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f60c80e1ef0 con 0x7f60d0109580 2024-06-02T15:41:53.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.226+0000 7f60cf400700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 0x7f60b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:53.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.226+0000 7f60cf400700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 0x7f60b40b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009680 tx=0x7f60c0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:53.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.229+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60c8022060 con 0x7f60d0109580 2024-06-02T15:41:53.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:52 smithi120 ceph-mon[36451]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:53.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:52 smithi135 ceph-mon[36839]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:41:53.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.512+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d01a8c10 con 0x7f60d0109580 2024-06-02T15:41:53.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.513+0000 7f60c7e00700 1 -- 172.21.15.90:0/1932581360 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f60c8010040 con 0x7f60d0109580 2024-06-02T15:41:53.511 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:41:53.511 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 msgr2=0x7f60b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 0x7f60b40b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009680 tx=0x7f60c0019040 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 msgr2=0x7f60d01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d01a0790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f60c800efc0 tx=0x7f60c800c3a0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 shutdown_connections 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60b40b2900 0x7f60b40b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60d0109580 0x7f60d01a0790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60d006bab0 0x7f60d019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 --2- 172.21.15.90:0/1932581360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60d006c470 0x7f60d019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.517+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 >> 172.21.15.90:0/1932581360 conn(0x7f60d0079280 msgr2=0x7f60d010e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.518+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 shutdown_connections 2024-06-02T15:41:53.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:53.518+0000 7f60d5000700 1 -- 172.21.15.90:0/1932581360 wait complete. 2024-06-02T15:41:53.516 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:41:54.208 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:53 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1932581360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:54.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:53 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1932581360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:54.300 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:41:54.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:53 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1932581360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:54.743 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:55.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:54 smithi090 ceph-mon[31595]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:55.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:54 smithi120 ceph-mon[36451]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:55.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:54 smithi135 ceph-mon[36839]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 -- 172.21.15.90:0/239463546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a4104690 msgr2=0x7f35a4104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 --2- 172.21.15.90:0/239463546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a4104690 0x7f35a4104a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3594009990 tx=0x7f3594009ca0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 -- 172.21.15.90:0/239463546 shutdown_connections 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 --2- 172.21.15.90:0/239463546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a410da70 0x7f35a410ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 --2- 172.21.15.90:0/239463546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a4104690 0x7f35a4104a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 --2- 172.21.15.90:0/239463546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104fb0 0x7f35a410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.987+0000 7f35a3e00700 1 -- 172.21.15.90:0/239463546 >> 172.21.15.90:0/239463546 conn(0x7f35a40fde60 msgr2=0x7f35a4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.988+0000 7f35a3e00700 1 -- 172.21.15.90:0/239463546 shutdown_connections 2024-06-02T15:41:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.988+0000 7f35a3e00700 1 -- 172.21.15.90:0/239463546 wait complete. 2024-06-02T15:41:56.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:56 smithi090 ceph-mon[31595]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.989+0000 7f35a3e00700 1 Processor -- start 2024-06-02T15:41:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.989+0000 7f35a3e00700 1 -- start start 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104690 0x7f35a41022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 0x7f35a4102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4113690 con 0x7f35a410da70 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4113510 con 0x7f35a4104fb0 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35a4113810 con 0x7f35a4104690 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.990+0000 7f35a3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 0x7f35a4102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f35a2a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104690 0x7f35a41022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:56.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f35a3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46126/0 (socket says 172.21.15.90:46126) 2024-06-02T15:41:56.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f35a3400700 1 -- 172.21.15.90:0/2209432678 learned_addr learned my addr 172.21.15.90:0/2209432678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 -- 172.21.15.90:0/2209432678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104690 msgr2=0x7f35a41022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104690 0x7f35a41022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 -- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 msgr2=0x7f35a4102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f359be00700 1 -- 172.21.15.90:0/2209432678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a41a80d0 con 0x7f35a4104fb0 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.991+0000 7f35a3400700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f359be00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 0x7f35a4102830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f358c00e9c0 tx=0x7f358c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c00c9c0 con 0x7f35a4104fb0 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358c00cb60 con 0x7f35a4104fb0 2024-06-02T15:41:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a41a83c0 con 0x7f35a4104fb0 2024-06-02T15:41:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c018680 con 0x7f35a4104fb0 2024-06-02T15:41:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.992+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a41a8950 con 0x7f35a4104fb0 2024-06-02T15:41:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.993+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a406a230 con 0x7f35a4104fb0 2024-06-02T15:41:56.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.995+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f358c04aab0 con 0x7f35a4104fb0 2024-06-02T15:41:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.997+0000 7f359b400700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:41:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.997+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f358c0e2880 con 0x7f35a4104fb0 2024-06-02T15:41:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.997+0000 7f35a2a00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:41:56.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.998+0000 7f35a2a00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 0x7f35880b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f359400b2e0 tx=0x7f3594005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:41:56.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:56.999+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f358c0aa190 con 0x7f35a4104fb0 2024-06-02T15:41:57.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.286+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35a4103820 con 0x7f35a4104fb0 2024-06-02T15:41:57.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.287+0000 7f359b400700 1 -- 172.21.15.90:0/2209432678 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f358c0a5070 con 0x7f35a4104fb0 2024-06-02T15:41:57.285 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:41:57.285 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:41:57.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 msgr2=0x7f35880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:57.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 0x7f35880b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f359400b2e0 tx=0x7f3594005fd0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 msgr2=0x7f35a4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:41:57.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 0x7f35a4102830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f358c00e9c0 tx=0x7f358c00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 shutdown_connections 2024-06-02T15:41:57.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f35880b2900 0x7f35880b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a410da70 0x7f35a4102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35a4104690 0x7f35a41022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 --2- 172.21.15.90:0/2209432678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f35a4104fb0 0x7f35a4102830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:41:57.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.292+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 >> 172.21.15.90:0/2209432678 conn(0x7f35a40fde60 msgr2=0x7f35a40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:41:57.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.293+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 shutdown_connections 2024-06-02T15:41:57.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:41:57.293+0000 7f35a3e00700 1 -- 172.21.15.90:0/2209432678 wait complete. 2024-06-02T15:41:57.291 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:41:57.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:56 smithi120 ceph-mon[36451]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:57.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:56 smithi135 ceph-mon[36839]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:41:58.240 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:57 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2209432678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:58.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:41:58.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:57 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2209432678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:58.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:57 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2209432678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:41:58.681 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:41:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:41:58 smithi090 ceph-mon[31595]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:59.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:41:58 smithi120 ceph-mon[36451]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:41:59.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:41:58 smithi135 ceph-mon[36839]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:00.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.982+0000 7f789e400700 1 -- 172.21.15.90:0/300893242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7898105770 msgr2=0x7f7898105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.982+0000 7f789e400700 1 --2- 172.21.15.90:0/300893242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7898105770 0x7f7898105bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7884009990 tx=0x7f7884009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 -- 172.21.15.90:0/300893242 shutdown_connections 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 --2- 172.21.15.90:0/300893242 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f7898112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 --2- 172.21.15.90:0/300893242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f789810b770 0x7f789810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 --2- 172.21.15.90:0/300893242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7898105770 0x7f7898105bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 -- 172.21.15.90:0/300893242 >> 172.21.15.90:0/300893242 conn(0x7f7898101240 msgr2=0x7f7898103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 -- 172.21.15.90:0/300893242 shutdown_connections 2024-06-02T15:42:00.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.983+0000 7f789e400700 1 -- 172.21.15.90:0/300893242 wait complete. 2024-06-02T15:42:00.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.984+0000 7f789e400700 1 Processor -- start 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.984+0000 7f789e400700 1 -- start start 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7898105770 0x7f789819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f789819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7898115a00 con 0x7f789810b770 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7898115880 con 0x7f7898106130 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f789e400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7898115b80 con 0x7f7898105770 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7897e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7896a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f789819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7897e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56476/0 (socket says 172.21.15.90:56476) 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7897e00700 1 -- 172.21.15.90:0/1540729985 learned_addr learned my addr 172.21.15.90:0/1540729985 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7897400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7898105770 0x7f789819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:00.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7896a00700 1 -- 172.21.15.90:0/1540729985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7898105770 msgr2=0x7f789819bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7896a00700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7898105770 0x7f789819bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.985+0000 7f7896a00700 1 -- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 msgr2=0x7f78981a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f7896a00700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f7896a00700 1 -- 172.21.15.90:0/1540729985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78981a0eb0 con 0x7f7898106130 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f7897e00700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f7896a00700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f789819c430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f7884009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884026070 con 0x7f7898106130 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78981a1140 con 0x7f7898106130 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78981a8a20 con 0x7f7898106130 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7884021410 con 0x7f7898106130 2024-06-02T15:42:00.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.986+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840175f0 con 0x7f7898106130 2024-06-02T15:42:00.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.987+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7878005330 con 0x7f7898106130 2024-06-02T15:42:00.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.989+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7884029070 con 0x7f7898106130 2024-06-02T15:42:00.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.991+0000 7f788fe00700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 0x7f787c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:00.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.991+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f78840e3100 con 0x7f7898106130 2024-06-02T15:42:00.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.991+0000 7f7897400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 0x7f787c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:00.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.992+0000 7f7897400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 0x7f787c0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7888009be0 tx=0x7f7888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:00.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:00.995+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7884013070 con 0x7f7898106130 2024-06-02T15:42:01.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:00 smithi090 ceph-mon[31595]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:01.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.236+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7878005620 con 0x7f7898106130 2024-06-02T15:42:01.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.238+0000 7f788fe00700 1 -- 172.21.15.90:0/1540729985 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7f78840aaaa0 con 0x7f7898106130 2024-06-02T15:42:01.236 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:01.237 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:42:01.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 msgr2=0x7f787c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 0x7f787c0b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7888009be0 tx=0x7f7888009260 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 msgr2=0x7f789819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f789819c430 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f7884009580 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 shutdown_connections 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f787c0b2880 0x7f787c0b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f789810b770 0x7f78981a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7898105770 0x7f789819bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 --2- 172.21.15.90:0/1540729985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7898106130 0x7f789819c430 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 >> 172.21.15.90:0/1540729985 conn(0x7f7898101240 msgr2=0x7f7898101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 shutdown_connections 2024-06-02T15:42:01.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:01.242+0000 7f789e400700 1 -- 172.21.15.90:0/1540729985 wait complete. 2024-06-02T15:42:01.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:00 smithi120 ceph-mon[36451]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:01.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:00 smithi135 ceph-mon[36839]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:02.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:42:02.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:01 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1540729985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:02.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:01 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1540729985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:02.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:01 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1540729985' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:02.677 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:03.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:02 smithi090 ceph-mon[31595]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:03.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:02 smithi120 ceph-mon[36451]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:03.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:02 smithi135 ceph-mon[36839]: pgmap v442: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.871+0000 7f8437e00700 1 -- 172.21.15.90:0/1723989681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8438104fc0 msgr2=0x7f843810d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.871+0000 7f8437e00700 1 --2- 172.21.15.90:0/1723989681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8438104fc0 0x7f843810d4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8420009990 tx=0x7f8420009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 -- 172.21.15.90:0/1723989681 shutdown_connections 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 --2- 172.21.15.90:0/1723989681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f843810dad0 0x7f843810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 --2- 172.21.15.90:0/1723989681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84381046a0 0x7f8438104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 --2- 172.21.15.90:0/1723989681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8438104fc0 0x7f843810d4b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 -- 172.21.15.90:0/1723989681 >> 172.21.15.90:0/1723989681 conn(0x7f84380fdef0 msgr2=0x7f8438100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:04.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 -- 172.21.15.90:0/1723989681 shutdown_connections 2024-06-02T15:42:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.872+0000 7f8437e00700 1 -- 172.21.15.90:0/1723989681 wait complete. 2024-06-02T15:42:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 Processor -- start 2024-06-02T15:42:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 -- start start 2024-06-02T15:42:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8438104fc0 0x7f843819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f843810dad0 0x7f84381a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113710 con 0x7f84381046a0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113590 con 0x7f843810dad0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.873+0000 7f8437e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8438113890 con 0x7f8438104fc0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8437400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f843810dad0 0x7f84381a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56496/0 (socket says 172.21.15.90:56496) 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 -- 172.21.15.90:0/161394508 learned_addr learned my addr 172.21.15.90:0/161394508 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f842e000700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8438104fc0 0x7f843819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 -- 172.21.15.90:0/161394508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8438104fc0 msgr2=0x7f843819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8438104fc0 0x7f843819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 -- 172.21.15.90:0/161394508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f843810dad0 msgr2=0x7f84381a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f843810dad0 0x7f84381a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.874+0000 7f8436a00700 1 -- 172.21.15.90:0/161394508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84381a0e10 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f8436a00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9a0 tx=0x7f842800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800ca10 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842800cbb0 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84381a1100 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428018680 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.875+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a8980 con 0x7f84381046a0 2024-06-02T15:42:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.876+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843806a230 con 0x7f84381046a0 2024-06-02T15:42:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.878+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8428010430 con 0x7f84381046a0 2024-06-02T15:42:04.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.880+0000 7f843ca00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 0x7f841c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:04.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.880+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f84280e2770 con 0x7f84381046a0 2024-06-02T15:42:04.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.880+0000 7f842e000700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 0x7f841c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:04.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.881+0000 7f842e000700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 0x7f841c0b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8438110420 tx=0x7f8420005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:04.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:04.885+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84280aa080 con 0x7f84381046a0 2024-06-02T15:42:05.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.169+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84381a8b50 con 0x7f84381046a0 2024-06-02T15:42:05.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.170+0000 7f843ca00700 1 -- 172.21.15.90:0/161394508 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f84280b2100 con 0x7f84381046a0 2024-06-02T15:42:05.167 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:05.168 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:42:05.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 msgr2=0x7f841c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:05.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 0x7f841c0b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8438110420 tx=0x7f8420005fd0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 msgr2=0x7f843819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f842800e9a0 tx=0x7f842800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 shutdown_connections 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f841c0b2880 0x7f841c0b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f843810dad0 0x7f84381a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84381046a0 0x7f843819be00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 --2- 172.21.15.90:0/161394508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8438104fc0 0x7f843819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 >> 172.21.15.90:0/161394508 conn(0x7f84380fdef0 msgr2=0x7f843810ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 shutdown_connections 2024-06-02T15:42:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:05.173+0000 7f8437e00700 1 -- 172.21.15.90:0/161394508 wait complete. 2024-06-02T15:42:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:42:05.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:04 smithi120 ceph-mon[36451]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:05.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:04 smithi135 ceph-mon[36839]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:05.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:04 smithi090 ceph-mon[31595]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:06.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-02T15:38:19.378346+0000', 'modified': '2024-06-02T15:39:40.241359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24524, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/3465448684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3465448684}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 3465448684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6839/3736093015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 3736093015}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 3736093015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6837/1676077338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1676077338}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1676077338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'g', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.90:6839/1414255295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 1414255295}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 1414255295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.90:6841/1722073812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 1722073812}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 1722073812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.120:6839/3642613472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 3642613472}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 3642613472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24536, 24512, 24524]} 2024-06-02T15:42:06.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-02T15:42:06.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.g status 2024-06-02T15:42:06.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:06 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/161394508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:06.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:06 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/161394508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:06.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:06 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/161394508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:07.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:07 smithi090 ceph-mon[31595]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:07.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:07 smithi120 ceph-mon[36451]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:07.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:07 smithi135 ceph-mon[36839]: pgmap v444: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 -- 172.21.15.90:0/3593367553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa10075770 msgr2=0x7ffa10075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 --2- 172.21.15.90:0/3593367553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa10075770 0x7ffa10075b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009990 tx=0x7ffa00009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 -- 172.21.15.90:0/3593367553 shutdown_connections 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 --2- 172.21.15.90:0/3593367553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa1010ffa0 0x7ffa10112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 --2- 172.21.15.90:0/3593367553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa10075770 0x7ffa10075b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 --2- 172.21.15.90:0/3593367553 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10076090 0x7ffa1010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 -- 172.21.15.90:0/3593367553 >> 172.21.15.90:0/3593367553 conn(0x7ffa1007a6a0 msgr2=0x7ffa1007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 -- 172.21.15.90:0/3593367553 shutdown_connections 2024-06-02T15:42:08.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.908+0000 7ffa16a00700 1 -- 172.21.15.90:0/3593367553 wait complete. 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.909+0000 7ffa16a00700 1 Processor -- start 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.909+0000 7ffa16a00700 1 -- start start 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa10076090 0x7ffa1019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1010ffa0 0x7ffa101a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffa101159a0 con 0x7ffa1010ffa0 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10115820 con 0x7ffa10075770 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa16a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10115b20 con 0x7ffa10076090 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:48010/0 (socket says 172.21.15.90:48010) 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa0f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa10076090 0x7ffa1019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa0fe00700 1 -- 172.21.15.90:0/4060170443 learned_addr learned my addr 172.21.15.90:0/4060170443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.910+0000 7ffa14a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1010ffa0 0x7ffa101a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 -- 172.21.15.90:0/4060170443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa10076090 msgr2=0x7ffa1019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa10076090 0x7ffa1019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 -- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1010ffa0 msgr2=0x7ffa101a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1010ffa0 0x7ffa101a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 -- 172.21.15.90:0/4060170443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101a0df0 con 0x7ffa10075770 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005910 tx=0x7ffa0000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0cc00700 1 -- 172.21.15.90:0/4060170443 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00027070 con 0x7ffa10075770 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa101a1080 con 0x7ffa10075770 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0cc00700 1 -- 172.21.15.90:0/4060170443 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa00022470 con 0x7ffa10075770 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.911+0000 7ffa0cc00700 1 -- 172.21.15.90:0/4060170443 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00017650 con 0x7ffa10075770 2024-06-02T15:42:08.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.912+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa101a8960 con 0x7ffa10075770 2024-06-02T15:42:08.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.915+0000 7ffa0cc00700 1 -- 172.21.15.90:0/4060170443 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa0002a070 con 0x7ffa10075770 2024-06-02T15:42:08.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.916+0000 7ffa213f5b80 1 Processor -- start 2024-06-02T15:42:08.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.916+0000 7ffa213f5b80 1 -- start start 2024-06-02T15:42:08.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.916+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 0x5586cfcec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.916+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586cfc44df0 con 0x5586cfcebe60 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586cfc34990 con 0x5586cfce6830 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586cfc389d0 con 0x5586cfcec780 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa0cc00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 0x7ff9f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa14a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 0x5586cfcec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa14a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 0x5586cfcec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56528/0 (socket says 172.21.15.90:56528) 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa14a00700 1 -- 172.21.15.90:0/2737280377 learned_addr learned my addr 172.21.15.90:0/2737280377 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:34780/0 (socket says 172.21.15.90:34780) 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa0cc00700 1 -- 172.21.15.90:0/4060170443 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffa000e32c0 con 0x7ffa10075770 2024-06-02T15:42:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa14a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 0x7ff9f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.917+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2737280377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 msgr2=0x5586cfce6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2737280377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 msgr2=0x5586cfcec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 0x5586cfcec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2737280377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cfbfd3d0 con 0x5586cfcec780 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa000aa260 tx=0x7ffa0009ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c56bf1074ddd9f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa14a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 0x7ff9f40b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e10 tx=0x7ffa0800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa05600700 1 -- 172.21.15.90:0/2737280377 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00031040 con 0x5586cfcec780 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa05600700 1 -- 172.21.15.90:0/2737280377 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000aa070 con 0x5586cfcec780 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.918+0000 7ffa05600700 1 -- 172.21.15.90:0/2737280377 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000b3330 con 0x5586cfcec780 2024-06-02T15:42:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2737280377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 msgr2=0x5586cfce6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa000aa260 tx=0x7ffa0009ee30 comp rx=0 tx=0).stop 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2737280377 shutdown_connections 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcebe60 0x5586cfcec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2737280377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcec780 0x5586cfce6210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2737280377 >> 172.21.15.90:0/2737280377 conn(0x5586cfc14b90 msgr2=0x5586cfce0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2737280377 shutdown_connections 2024-06-02T15:42:08.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.919+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2737280377 wait complete. 2024-06-02T15:42:08.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.920+0000 7ffa213f5b80 1 -- start start 2024-06-02T15:42:08.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.920+0000 7ffa213f5b80 1 Processor -- start 2024-06-02T15:42:08.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfce6830 0x5586cfcf2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfcebe60 0x5586cfcef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586cfb3af50 con 0x5586cfcec780 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586cfbd5710 con 0x5586cfcebe60 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586cfcf2fb0 con 0x5586cfce6830 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa213f5b80 10 client.0 _ll_register_callbacks cb 0x5586cfbdec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56540/0 (socket says 172.21.15.90:56540) 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 -- 172.21.15.90:0/3484834492 learned_addr learned my addr 172.21.15.90:0/3484834492 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa0f400700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfcebe60 0x5586cfcef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfce6830 0x5586cfcf2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 -- 172.21.15.90:0/3484834492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfce6830 msgr2=0x5586cfcf2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfce6830 0x5586cfcf2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 -- 172.21.15.90:0/3484834492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfcebe60 msgr2=0x5586cfcef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfcebe60 0x5586cfcef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.921+0000 7ffa14a00700 1 -- 172.21.15.90:0/3484834492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cfb84880 con 0x5586cfcec780 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ffa14a00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffa0801ad00 tx=0x7ffa08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75c83b978295d1b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ff9fa000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08004bb0 con 0x5586cfcec780 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffa08012040 con 0x5586cfcec780 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa000aa070 con 0x5586cfcec780 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa08031050 con 0x5586cfcec780 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ffa213f5b80 10 client.14859 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:08.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.922+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586cfc05010 con 0x5586cfcec780 2024-06-02T15:42:08.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000aa070 con 0x5586cfcec780 2024-06-02T15:42:08.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa0801b4e0 con 0x5586cfcec780 2024-06-02T15:42:08.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 10 client.14859 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:08.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 10 client.14859 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-02T15:42:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 20 client.14859 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcffa00 0x5586cfd00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 4 client.14859 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:42:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.923+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586cfc46d40 con 0x5586cfcffa00 2024-06-02T15:42:08.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.924+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcffa00 0x5586cfd00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.924+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcffa00 0x5586cfd00e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5d4809e2d44147 server_cookie=542b4b47fc0af7bf in_seq=0 out_seq=0 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.924+0000 7ff9fa000700 10 client.14859 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ff9fa000700 1 -- 172.21.15.90:0/3484834492 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5586cfc46d40 con 0x5586cfcffa00 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ff9fa000700 10 client.14859 handle_command_reply: tid=0 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 client.14859 shutdown 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 msgr2=0x5586cfcefd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffa0801ad00 tx=0x7ffa08004ad0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 shutdown_connections 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcffa00 0x5586cfd00e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfcec780 0x5586cfcefd30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfce6830 0x5586cfcf2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/3484834492 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfcebe60 0x5586cfcef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ff9fa000700 0 client.14859 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.925+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 >> 172.21.15.90:0/3484834492 conn(0x5586cfc14b90 msgr2=0x5586cfce0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.926+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 shutdown_connections 2024-06-02T15:42:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.926+0000 7ffa213f5b80 1 -- 172.21.15.90:0/3484834492 wait complete. 2024-06-02T15:42:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.926+0000 7ffa213f5b80 20 client.14859 trim_cache size 0 max 16384 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.949+0000 7ffa213f5b80 1 Processor -- start 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.949+0000 7ffa213f5b80 1 -- start start 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 0x5586cfce0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586cfcff1b0 con 0x5586cfc14b90 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586cfcff880 con 0x5586cfce6830 2024-06-02T15:42:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586cfcfff50 con 0x5586cfcebe60 2024-06-02T15:42:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 0x5586cfce0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 0x5586cfce0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56546/0 (socket says 172.21.15.90:56546) 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2221227783 learned_addr learned my addr 172.21.15.90:0/2221227783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa14a00700 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.950+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 -- 172.21.15.90:0/2221227783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 msgr2=0x5586cfd0a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd0a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 -- 172.21.15.90:0/2221227783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 msgr2=0x5586cfce0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 0x5586cfce0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 -- 172.21.15.90:0/2221227783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586cfcedd60 con 0x5586cfce6830 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce0ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5586cfcf0dc0 tx=0x7ff9fc010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b185daebcfbacf81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa04c00700 1 -- 172.21.15.90:0/2221227783 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc010b60 con 0x5586cfce6830 2024-06-02T15:42:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa04c00700 1 -- 172.21.15.90:0/2221227783 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc010d00 con 0x5586cfce6830 2024-06-02T15:42:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.951+0000 7ffa04c00700 1 -- 172.21.15.90:0/2221227783 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc005640 con 0x5586cfce6830 2024-06-02T15:42:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2221227783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 msgr2=0x5586cfce0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce0ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5586cfcf0dc0 tx=0x7ff9fc010710 comp rx=0 tx=0).stop 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2221227783 shutdown_connections 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd0a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfc14b90 0x5586cfce0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2221227783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfce6830 0x5586cfce0ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2221227783 >> 172.21.15.90:0/2221227783 conn(0x5586cfcf27b0 msgr2=0x5586cfce9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2221227783 shutdown_connections 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.952+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2221227783 wait complete. 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.953+0000 7ffa213f5b80 1 -- start start 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.953+0000 7ffa213f5b80 1 Processor -- start 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 0x5586cfd0aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586cfd00cf0 con 0x5586cfce6830 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5586cfd00620 con 0x5586cfc14b90 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586cfd013c0 con 0x5586cfcebe60 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa213f5b80 10 client.0 _ll_register_callbacks cb 0x5586cfbdec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:48034/0 (socket says 172.21.15.90:48034) 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 0x5586cfd0aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2905625036 learned_addr learned my addr 172.21.15.90:0/2905625036 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa14a00700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2905625036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 msgr2=0x5586cfd054a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd054a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 msgr2=0x5586cfd0aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 0x5586cfd0aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0fe00700 1 -- 172.21.15.90:0/2905625036 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586cfd05ce0 con 0x5586cfc14b90 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.954+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 0x5586cfd0aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ffa0fe00700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00022cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155d3f0a6436f01a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ff9fa000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000a9eb0 con 0x5586cfc14b90 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc010d00 con 0x5586cfc14b90 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffa000aa7c0 con 0x5586cfc14b90 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa000aa070 con 0x5586cfc14b90 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ffa213f5b80 10 client.24983 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.955+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586cfd74650 con 0x5586cfc14b90 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc010d00 con 0x5586cfc14b90 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa000b2ea0 con 0x5586cfc14b90 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 10 client.24983 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 10 client.24983 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 20 client.24983 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcec780 0x5586cfd77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 4 client.24983 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "status"}) -- 0x5586cfc40520 con 0x5586cfcec780 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.956+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcec780 0x5586cfd77d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa0f400700 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcec780 0x5586cfd77d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b582351318ced7dd server_cookie=c0c7c9ba75fdd3ff in_seq=0 out_seq=0 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ff9fa000700 10 client.24983 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ff9fa000700 1 -- 172.21.15.90:0/2905625036 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5586cfc40520 con 0x5586cfcec780 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ff9fa000700 10 client.24983 handle_command_reply: tid=0 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 client.24983 shutdown 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 msgr2=0x5586cfd0a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffa00000c00 tx=0x7ffa00022cc0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 shutdown_connections 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5586cfcec780 0x5586cfd77d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586cfcebe60 0x5586cfd054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5586cfc14b90 0x5586cfd0a5b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ff9fa000700 0 client.24983 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 --2- 172.21.15.90:0/2905625036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586cfce6830 0x5586cfd0aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.957+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 >> 172.21.15.90:0/2905625036 conn(0x5586cfcf27b0 msgr2=0x5586cfce0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.958+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 shutdown_connections 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.958+0000 7ffa213f5b80 1 -- 172.21.15.90:0/2905625036 wait complete. 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.958+0000 7ffa213f5b80 20 client.24983 trim_cache size 0 max 16384 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24512, 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:42:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 156.05424866000001, 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 259.52694414600001 2024-06-02T15:42:08.960 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 msgr2=0x7ff9f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 0x7ff9f40b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e10 tx=0x7ffa0800b040 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 msgr2=0x7ffa1019be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa00005910 tx=0x7ffa0000b670 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 shutdown_connections 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9f40b2880 0x7ff9f40b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1010ffa0 0x7ffa101a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa10075770 0x7ffa1019be30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 --2- 172.21.15.90:0/4060170443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffa10076090 0x7ffa1019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 >> 172.21.15.90:0/4060170443 conn(0x7ffa1007a6a0 msgr2=0x7ffa101007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 shutdown_connections 2024-06-02T15:42:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:08.959+0000 7ffa16a00700 1 -- 172.21.15.90:0/4060170443 wait complete. 2024-06-02T15:42:09.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:09 smithi120 ceph-mon[36451]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:09.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:09 smithi135 ceph-mon[36839]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:09 smithi090 ceph-mon[31595]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:09.794 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.a status 2024-06-02T15:42:10.246 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:11.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:11 smithi120 ceph-mon[36451]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:11.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:11 smithi135 ceph-mon[36839]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:11.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:11 smithi090 ceph-mon[31595]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 -- 172.21.15.90:0/3813356733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634076090 msgr2=0x7fa63410fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 --2- 172.21.15.90:0/3813356733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634076090 0x7fa63410fa00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa620009990 tx=0x7fa620009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 -- 172.21.15.90:0/3813356733 shutdown_connections 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 --2- 172.21.15.90:0/3813356733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa63410ff40 0x7fa634112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 --2- 172.21.15.90:0/3813356733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa634075770 0x7fa634075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 --2- 172.21.15.90:0/3813356733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634076090 0x7fa63410fa00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 -- 172.21.15.90:0/3813356733 >> 172.21.15.90:0/3813356733 conn(0x7fa63407a6a0 msgr2=0x7fa63407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 -- 172.21.15.90:0/3813356733 shutdown_connections 2024-06-02T15:42:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.508+0000 7fa638c00700 1 -- 172.21.15.90:0/3813356733 wait complete. 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.509+0000 7fa638c00700 1 Processor -- start 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 -- start start 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 0x7fa63419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 0x7fa6341a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa634115950 con 0x7fa634075770 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6341157d0 con 0x7fa634076090 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa638c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa634115ad0 con 0x7fa63410ff40 2024-06-02T15:42:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa631e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa631e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44092/0 (socket says 172.21.15.90:44092) 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa631e00700 1 -- 172.21.15.90:0/4114571018 learned_addr learned my addr 172.21.15.90:0/4114571018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa631400700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 0x7fa63419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.510+0000 7fa632800700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 0x7fa6341a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 -- 172.21.15.90:0/4114571018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 msgr2=0x7fa6341a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 0x7fa6341a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 -- 172.21.15.90:0/4114571018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 msgr2=0x7fa63419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 0x7fa63419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 -- 172.21.15.90:0/4114571018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341a0e50 con 0x7fa634075770 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631400700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 0x7fa63419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa632800700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 0x7fa6341a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:42:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa631e00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e9a0 tx=0x7fa62400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa62aa00700 1 -- 172.21.15.90:0/4114571018 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62400cbd0 con 0x7fa634075770 2024-06-02T15:42:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a1140 con 0x7fa634075770 2024-06-02T15:42:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.511+0000 7fa62aa00700 1 -- 172.21.15.90:0/4114571018 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62400cd70 con 0x7fa634075770 2024-06-02T15:42:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.512+0000 7fa62aa00700 1 -- 172.21.15.90:0/4114571018 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624018680 con 0x7fa634075770 2024-06-02T15:42:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.512+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a89c0 con 0x7fa634075770 2024-06-02T15:42:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.515+0000 7fa62aa00700 1 -- 172.21.15.90:0/4114571018 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa624010430 con 0x7fa634075770 2024-06-02T15:42:12.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa64365eb80 1 Processor -- start 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa62aa00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa64365eb80 1 -- start start 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa62aa00700 1 -- 172.21.15.90:0/4114571018 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa6240e28a0 con 0x7fa634075770 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa632800700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfadf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.517+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfada570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa64365eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfa37df0 con 0x55dbcfada0f0 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa64365eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfa27990 con 0x55dbcfadfee0 2024-06-02T15:42:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa64365eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfa2b9d0 con 0x55dbcfadf5c0 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa632800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfada570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfadf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa632800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfada570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44102/0 (socket says 172.21.15.90:44102) 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37114/0 (socket says 172.21.15.90:37114) 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa632800700 1 -- 172.21.15.90:0/3871899720 learned_addr learned my addr 172.21.15.90:0/3871899720 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfadf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41018/0 (socket says 172.21.15.90:41018) 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 -- 172.21.15.90:0/3871899720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 msgr2=0x55dbcfadf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfadf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 -- 172.21.15.90:0/3871899720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 msgr2=0x55dbcfada570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.518+0000 7fa631400700 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfada570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa631400700 1 -- 172.21.15.90:0/3871899720 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbcf9f03d0 con 0x55dbcfadfee0 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa632800700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 0x7fa6180b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0077d0 tx=0x7fa62c007b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa631400700 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa620000c00 tx=0x7fa620005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc25bc5e328b1b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa61f400700 1 -- 172.21.15.90:0/3871899720 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620027070 con 0x55dbcfadfee0 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa61f400700 1 -- 172.21.15.90:0/3871899720 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620003680 con 0x55dbcfadfee0 2024-06-02T15:42:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.519+0000 7fa61f400700 1 -- 172.21.15.90:0/3871899720 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62000f6c0 con 0x55dbcfadfee0 2024-06-02T15:42:12.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 -- 172.21.15.90:0/3871899720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 msgr2=0x55dbcfad9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa620000c00 tx=0x7fa620005fd0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 -- 172.21.15.90:0/3871899720 shutdown_connections 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfada570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfadf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 --2- 172.21.15.90:0/3871899720 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadfee0 0x55dbcfad9ad0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 -- 172.21.15.90:0/3871899720 >> 172.21.15.90:0/3871899720 conn(0x55dbcfa07b90 msgr2=0x55dbcfad4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 -- 172.21.15.90:0/3871899720 shutdown_connections 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.520+0000 7fa64365eb80 1 -- 172.21.15.90:0/3871899720 wait complete. 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.521+0000 7fa64365eb80 1 -- start start 2024-06-02T15:42:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.521+0000 7fa64365eb80 1 Processor -- start 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.521+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 0x55dbcfae3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfae3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dbcf92df50 con 0x55dbcfada0f0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbcf9c8710 con 0x55dbcfadf5c0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfae6930 con 0x55dbcfadfee0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa64365eb80 10 client.0 _ll_register_callbacks cb 0x55dbcf9d1c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44106/0 (socket says 172.21.15.90:44106) 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631e00700 1 -- 172.21.15.90:0/222004946 learned_addr learned my addr 172.21.15.90:0/222004946 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 0x55dbcfae3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa632800700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfae3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 -- 172.21.15.90:0/222004946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 msgr2=0x55dbcfae3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfae3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 -- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 msgr2=0x55dbcfae5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631400700 1 -- 172.21.15.90:0/222004946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbcf977880 con 0x55dbcfadf5c0 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.522+0000 7fa631e00700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa631400700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 0x55dbcfae3100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098e0 tx=0x7fa620003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1352bc2aeefdf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa613e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620027040 con 0x55dbcfadf5c0 2024-06-02T15:42:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa620003680 con 0x55dbcfadf5c0 2024-06-02T15:42:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa6200050f0 con 0x55dbcfadf5c0 2024-06-02T15:42:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa62003a050 con 0x55dbcfadf5c0 2024-06-02T15:42:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa64365eb80 10 client.24995 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.523+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbcf9f8010 con 0x55dbcfadf5c0 2024-06-02T15:42:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620003680 con 0x55dbcfadf5c0 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa62000fb60 con 0x55dbcfadf5c0 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 10 client.24995 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 10 client.24995 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 20 client.24995 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfaf22e0 0x55dbcfaf4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 4 client.24995 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbcfa39d40 con 0x55dbcfaf22e0 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.524+0000 7fa631e00700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfaf22e0 0x55dbcfaf4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.525+0000 7fa631e00700 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfaf22e0 0x55dbcfaf4440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8b6d0f5d1c9c84e server_cookie=bf6d59e3ca9e7974 in_seq=0 out_seq=0 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.525+0000 7fa613e00700 10 client.24995 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:42:12.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa613e00700 1 -- 172.21.15.90:0/222004946 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dbcfa39d40 con 0x55dbcfaf22e0 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa613e00700 10 client.24995 handle_command_reply: tid=0 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 client.24995 shutdown 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 msgr2=0x55dbcfae3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 0x55dbcfae3100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa6200098e0 tx=0x7fa620003e90 comp rx=0 tx=0).stop 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 shutdown_connections 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfae3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfada0f0 0x55dbcfae5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfaf22e0 0x55dbcfaf4440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 --2- 172.21.15.90:0/222004946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfadf5c0 0x55dbcfae3100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa613e00700 0 client.24995 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.526+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 >> 172.21.15.90:0/222004946 conn(0x55dbcfa07b90 msgr2=0x55dbcfad3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.527+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 shutdown_connections 2024-06-02T15:42:12.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.527+0000 7fa64365eb80 1 -- 172.21.15.90:0/222004946 wait complete. 2024-06-02T15:42:12.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.527+0000 7fa64365eb80 20 client.24995 trim_cache size 0 max 16384 2024-06-02T15:42:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.550+0000 7fa64365eb80 1 Processor -- start 2024-06-02T15:42:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.550+0000 7fa64365eb80 1 -- start start 2024-06-02T15:42:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.550+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfade930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.550+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadfee0 0x55dbcfae02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa64365eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf2780 con 0x55dbcfadfee0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa64365eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf2e50 con 0x55dbcfada0f0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa64365eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf3520 con 0x55dbcfadf5c0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41048/0 (socket says 172.21.15.90:41048) 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 -- 172.21.15.90:0/2041769594 learned_addr learned my addr 172.21.15.90:0/2041769594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa632800700 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadfee0 0x55dbcfae02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631e00700 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfade930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 -- 172.21.15.90:0/2041769594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 msgr2=0x55dbcfade930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfade930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 -- 172.21.15.90:0/2041769594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadfee0 msgr2=0x55dbcfae02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadfee0 0x55dbcfae02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.551+0000 7fa631400700 1 -- 172.21.15.90:0/2041769594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbcfadf1f0 con 0x55dbcfadf5c0 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.552+0000 7fa631400700 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa620009ce0 tx=0x7fa62002c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f380cca34bf69dac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.552+0000 7fa61ea00700 1 -- 172.21.15.90:0/2041769594 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620003680 con 0x55dbcfadf5c0 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.552+0000 7fa61ea00700 1 -- 172.21.15.90:0/2041769594 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620031070 con 0x55dbcfadf5c0 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.552+0000 7fa61ea00700 1 -- 172.21.15.90:0/2041769594 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62000f460 con 0x55dbcfadf5c0 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- 172.21.15.90:0/2041769594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 msgr2=0x55dbcfad44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa620009ce0 tx=0x7fa62002c7c0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- 172.21.15.90:0/2041769594 shutdown_connections 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadfee0 0x55dbcfae02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfade930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 --2- 172.21.15.90:0/2041769594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad44a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- 172.21.15.90:0/2041769594 >> 172.21.15.90:0/2041769594 conn(0x55dbcfa07b90 msgr2=0x55dbcfade220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- 172.21.15.90:0/2041769594 shutdown_connections 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- 172.21.15.90:0/2041769594 wait complete. 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.553+0000 7fa64365eb80 1 -- start start 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 Processor -- start 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfafddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf42c0 con 0x55dbcfadf5c0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf3bf0 con 0x55dbcfada0f0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dbcfaf4990 con 0x55dbcfadfee0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa64365eb80 10 client.0 _ll_register_callbacks cb 0x55dbcf9d1c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.554+0000 7fa631e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfafddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa631400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41054/0 (socket says 172.21.15.90:41054) 2024-06-02T15:42:12.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 -- 172.21.15.90:0/786706935 learned_addr learned my addr 172.21.15.90:0/786706935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:12.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa631e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfafddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37132/0 (socket says 172.21.15.90:37132) 2024-06-02T15:42:12.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 -- 172.21.15.90:0/786706935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 msgr2=0x55dbcfafddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfafddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 -- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadf5c0 msgr2=0x55dbcfad35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:12.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 -- 172.21.15.90:0/786706935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbcfad4120 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa632800700 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007f90 tx=0x7fa62c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6fcf29a357e535 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.555+0000 7fa613e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c01ac90 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa62c025480 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa620031070 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa62c036050 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa64365eb80 10 client.24970 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dbcfb66f70 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.556+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa620031070 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa62c018040 con 0x55dbcfadfee0 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 10 client.24970 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 10 client.24970 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 20 client.24970 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:12.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfb6ba50 0x55dbcfb6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 4 client.24970 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "status"}) -- 0x55dbcfa33520 con 0x55dbcfb6ba50 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.557+0000 7fa631e00700 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfb6ba50 0x55dbcfb6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa613e00700 10 client.24970 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa631e00700 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfb6ba50 0x55dbcfb6a760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12722de1427bfc42 server_cookie=1c30f1e031ec2af7 in_seq=0 out_seq=0 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa613e00700 1 -- 172.21.15.90:0/786706935 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dbcfa33520 con 0x55dbcfb6ba50 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa613e00700 10 client.24970 handle_command_reply: tid=0 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 client.24970 shutdown 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 msgr2=0x55dbcfad3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007f90 tx=0x7fa62c004b40 comp rx=0 tx=0).stop 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 shutdown_connections 2024-06-02T15:42:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dbcfadfee0 0x55dbcfad3b10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dbcfb6ba50 0x55dbcfb6a760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dbcfada0f0 0x55dbcfafddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 --2- 172.21.15.90:0/786706935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dbcfadf5c0 0x55dbcfad35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 >> 172.21.15.90:0/786706935 conn(0x55dbcfa07b90 msgr2=0x55dbcfadeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa613e00700 0 client.24970 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.558+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 shutdown_connections 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa64365eb80 1 -- 172.21.15.90:0/786706935 wait complete. 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa64365eb80 20 client.24970 trim_cache size 0 max 16384 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:42:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24524, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 160.66199167600001, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 258.41076796700003 2024-06-02T15:42:12.560 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 msgr2=0x7fa6180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 0x7fa6180b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0077d0 tx=0x7fa62c007b90 comp rx=0 tx=0).stop 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 msgr2=0x7fa63419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa62400e9a0 tx=0x7fa62400ecb0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 shutdown_connections 2024-06-02T15:42:12.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.559+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa63410ff40 0x7fa6341a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.560+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa634075770 0x7fa63419be40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.560+0000 7fa638c00700 1 --2- 172.21.15.90:0/4114571018 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa634076090 0x7fa63419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.560+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 >> 172.21.15.90:0/4114571018 conn(0x7fa63407a6a0 msgr2=0x7fa6341007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.560+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 shutdown_connections 2024-06-02T15:42:12.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:12.560+0000 7fa638c00700 1 -- 172.21.15.90:0/4114571018 wait complete. 2024-06-02T15:42:13.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:13 smithi120 ceph-mon[36451]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:13.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:13 smithi135 ceph-mon[36839]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:13.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:13 smithi090 ceph-mon[31595]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:13.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T15:42:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:15.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:15 smithi120 ceph-mon[36451]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:15.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:15 smithi135 ceph-mon[36839]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:15.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:15 smithi090 ceph-mon[31595]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:16.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.286+0000 7ff9ca800700 1 -- 172.21.15.90:0/3699611177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c4105650 msgr2=0x7ff9c4105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.286+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3699611177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c4105650 0x7ff9c4105ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00098f0 tx=0x7ff9b0009c00 comp rx=0 tx=0).stop 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 -- 172.21.15.90:0/3699611177 shutdown_connections 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3699611177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4106010 0x7ff9c4112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3699611177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c410b650 0x7ff9c410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3699611177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c4105650 0x7ff9c4105ad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 -- 172.21.15.90:0/3699611177 >> 172.21.15.90:0/3699611177 conn(0x7ff9c4101180 msgr2=0x7ff9c41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 -- 172.21.15.90:0/3699611177 shutdown_connections 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.287+0000 7ff9ca800700 1 -- 172.21.15.90:0/3699611177 wait complete. 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.288+0000 7ff9ca800700 1 Processor -- start 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.288+0000 7ff9ca800700 1 -- start start 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.288+0000 7ff9ca800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 0x7ff9c419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.288+0000 7ff9ca800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 0x7ff9c419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.288+0000 7ff9ca800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 0x7ff9c41a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9ca800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4115830 con 0x7ff9c4106010 2024-06-02T15:42:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9ca800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41156b0 con 0x7ff9c4105650 2024-06-02T15:42:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9ca800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41159b0 con 0x7ff9c410b650 2024-06-02T15:42:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 0x7ff9c419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 0x7ff9c419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44138/0 (socket says 172.21.15.90:44138) 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c8a00700 1 -- 172.21.15.90:0/3897589602 learned_addr learned my addr 172.21.15.90:0/3897589602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 0x7ff9c41a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9400700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 0x7ff9c419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 -- 172.21.15.90:0/3897589602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 msgr2=0x7ff9c419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 0x7ff9c419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 -- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 msgr2=0x7ff9c419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 0x7ff9c419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9e00700 1 -- 172.21.15.90:0/3897589602 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a0d30 con 0x7ff9c410b650 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.289+0000 7ff9c9400700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 0x7ff9c419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 0x7ff9c41a05f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00efc0 tx=0x7ff9bc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9c2000700 1 -- 172.21.15.90:0/3897589602 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0092e0 con 0x7ff9c410b650 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9c2000700 1 -- 172.21.15.90:0/3897589602 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc004bb0 con 0x7ff9c410b650 2024-06-02T15:42:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a0fc0 con 0x7ff9c410b650 2024-06-02T15:42:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9c2000700 1 -- 172.21.15.90:0/3897589602 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc004020 con 0x7ff9c410b650 2024-06-02T15:42:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.290+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a88a0 con 0x7ff9c410b650 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.294+0000 7ff9c2000700 1 -- 172.21.15.90:0/3897589602 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff9bc01b030 con 0x7ff9c410b650 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9d682bb80 1 Processor -- start 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9d682bb80 1 -- start start 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9c2000700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 0x7ff9ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9c2000700 1 -- 172.21.15.90:0/3897589602 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9bc0e2070 con 0x7ff9c410b650 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.295+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c411df0 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c401990 con 0x55ef9c4b98f0 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4059d0 con 0x55ef9c4b3a80 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 0x7ff9ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44144/0 (socket says 172.21.15.90:44144) 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9400700 1 -- 172.21.15.90:0/1598059985 learned_addr learned my addr 172.21.15.90:0/1598059985 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 -- 172.21.15.90:0/1598059985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b98f0 msgr2=0x55ef9c4b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 -- 172.21.15.90:0/1598059985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 msgr2=0x55ef9c4b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9e00700 1 -- 172.21.15.90:0/1598059985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9c3ca3d0 con 0x55ef9c4b3a80 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.296+0000 7ff9c9400700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.297+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b3ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0df760 tx=0x7ff9bc0a8b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf80120a8465109 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.297+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 0x7ff9ac0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009680 tx=0x7ff9b000ff40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.297+0000 7ff9b6a00700 1 -- 172.21.15.90:0/1598059985 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0a9a10 con 0x55ef9c4b3a80 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.297+0000 7ff9b6a00700 1 -- 172.21.15.90:0/1598059985 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc0a9310 con 0x55ef9c4b3a80 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.297+0000 7ff9b6a00700 1 -- 172.21.15.90:0/1598059985 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0a4ee0 con 0x55ef9c4b3a80 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1598059985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 msgr2=0x55ef9c4b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b3ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0df760 tx=0x7ff9bc0a8b50 comp rx=0 tx=0).stop 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1598059985 shutdown_connections 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b3ee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1598059985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1598059985 >> 172.21.15.90:0/1598059985 conn(0x55ef9c3e1b90 msgr2=0x55ef9c4ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1598059985 shutdown_connections 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.298+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1598059985 wait complete. 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 -- start start 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 Processor -- start 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4bf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c307f50 con 0x55ef9c4b3a80 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c3a2710 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4c0410 con 0x55ef9c4b98f0 2024-06-02T15:42:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.299+0000 7ff9d682bb80 10 client.0 _ll_register_callbacks cb 0x55ef9c3abc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4bf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37186/0 (socket says 172.21.15.90:37186) 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 -- 172.21.15.90:0/1780437626 learned_addr learned my addr 172.21.15.90:0/1780437626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 -- 172.21.15.90:0/1780437626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 msgr2=0x55ef9c4bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 -- 172.21.15.90:0/1780437626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b3a80 msgr2=0x55ef9c4bf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4bf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 -- 172.21.15.90:0/1780437626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9c351880 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009230 tx=0x7ff9b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cdec2a8b602da3f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.300+0000 7ff9ab400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:42:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0020040 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9b0005640 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc0a9310 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9b0039050 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9d682bb80 10 client.25013 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef9c3d2010 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.301+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9bc0a9310 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9b000b7c0 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 10 client.25013 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 10 client.25013 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 20 client.25013 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c4ccfb0 0x55ef9c4ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 4 client.25013 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9c413d40 con 0x55ef9c4ccfb0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.302+0000 7ff9c9400700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c4ccfb0 0x55ef9c4ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.303+0000 7ff9c9400700 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c4ccfb0 0x55ef9c4ce420 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5e57af302798bd server_cookie=5cb902fffa0543b5 in_seq=0 out_seq=0 2024-06-02T15:42:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.303+0000 7ff9ab400700 10 client.25013 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.303+0000 7ff9ab400700 1 -- 172.21.15.90:0/1780437626 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ef9c413d40 con 0x55ef9c4ccfb0 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.303+0000 7ff9ab400700 10 client.25013 handle_command_reply: tid=0 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 client.25013 shutdown 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 msgr2=0x55ef9c4bca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009230 tx=0x7ff9b0009c00 comp rx=0 tx=0).stop 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 shutdown_connections 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c4ccfb0 0x55ef9c4ce420 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4bf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/1780437626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4bca00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 >> 172.21.15.90:0/1780437626 conn(0x55ef9c3e1b90 msgr2=0x55ef9c4b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9ab400700 0 client.25013 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:42:16.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 shutdown_connections 2024-06-02T15:42:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 1 -- 172.21.15.90:0/1780437626 wait complete. 2024-06-02T15:42:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.304+0000 7ff9d682bb80 20 client.25013 trim_cache size 0 max 16384 2024-06-02T15:42:16.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.327+0000 7ff9d682bb80 1 Processor -- start 2024-06-02T15:42:16.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.327+0000 7ff9d682bb80 1 -- start start 2024-06-02T15:42:16.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.327+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4cc760 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4cce30 con 0x55ef9c4b3a80 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4cd500 con 0x55ef9c4b98f0 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37188/0 (socket says 172.21.15.90:37188) 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 -- 172.21.15.90:0/470343759 learned_addr learned my addr 172.21.15.90:0/470343759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 -- 172.21.15.90:0/470343759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 msgr2=0x55ef9c4b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 -- 172.21.15.90:0/470343759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 msgr2=0x55ef9c4b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.328+0000 7ff9c9400700 1 -- 172.21.15.90:0/470343759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9c4ccc50 con 0x55ef9c4b3a80 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.329+0000 7ff9c9400700 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55ef9c4cef00 tx=0x7ff9b80045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a79c91fd92b183 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.329+0000 7ff9b6000700 1 -- 172.21.15.90:0/470343759 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8004830 con 0x55ef9c4b3a80 2024-06-02T15:42:16.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.329+0000 7ff9b6000700 1 -- 172.21.15.90:0/470343759 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b8005020 con 0x55ef9c4b3a80 2024-06-02T15:42:16.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.329+0000 7ff9b6000700 1 -- 172.21.15.90:0/470343759 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80038e0 con 0x55ef9c4b3a80 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- 172.21.15.90:0/470343759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 msgr2=0x55ef9c4b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55ef9c4cef00 tx=0x7ff9b80045f0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- 172.21.15.90:0/470343759 shutdown_connections 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4b2450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/470343759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- 172.21.15.90:0/470343759 >> 172.21.15.90:0/470343759 conn(0x55ef9c3e1b90 msgr2=0x55ef9c4cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- 172.21.15.90:0/470343759 shutdown_connections 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- 172.21.15.90:0/470343759 wait complete. 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.330+0000 7ff9d682bb80 1 -- start start 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 Processor -- start 2024-06-02T15:42:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4d7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4ce2a0 con 0x55ef9c4b8fd0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4cdbd0 con 0x55ef9c4b3a80 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef9c4ce970 con 0x55ef9c4b98f0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9d682bb80 10 client.0 _ll_register_callbacks cb 0x55ef9c3abc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4d7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.331+0000 7ff9c9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37192/0 (socket says 172.21.15.90:37192) 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9400700 1 -- 172.21.15.90:0/108249108 learned_addr learned my addr 172.21.15.90:0/108249108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 -- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 msgr2=0x55ef9c4d78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 -- 172.21.15.90:0/108249108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 msgr2=0x55ef9c4d7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4d7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 -- 172.21.15.90:0/108249108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9c4d2e90 con 0x55ef9c4b98f0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c8a00700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4d7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9400700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9c9e00700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4d2850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55ef9c4c00e0 tx=0x7ff9bc00c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd45843768c40197 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9ab400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.332+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0c2070 con 0x55ef9c4b98f0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b8005020 con 0x55ef9c4b98f0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9bc0a4c80 con 0x55ef9c4b98f0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc0a9a10 con 0x55ef9c4b98f0 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 10 client.24988 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef9c538fe0 con 0x55ef9c4b98f0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b8005020 con 0x55ef9c4b98f0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9bc0a8100 con 0x55ef9c4b98f0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 10 client.24988 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 10 client.24988 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 20 client.24988 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c53d670 0x55ef9c53c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 4 client.24988 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.333+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "status"}) -- 0x55ef9c40d520 con 0x55ef9c53d670 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.334+0000 7ff9c9400700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c53d670 0x55ef9c53c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.334+0000 7ff9c9400700 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c53d670 0x55ef9c53c380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e940fcec2e39a19d server_cookie=9356b4a775845198 in_seq=0 out_seq=0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.334+0000 7ff9ab400700 10 client.24988 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9ab400700 1 -- 172.21.15.90:0/108249108 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ef9c40d520 con 0x55ef9c53d670 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9ab400700 10 client.24988 handle_command_reply: tid=0 2024-06-02T15:42:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 client.24988 shutdown 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 msgr2=0x55ef9c4d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4d2850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55ef9c4c00e0 tx=0x7ff9bc00c9b0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 shutdown_connections 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ef9c53d670 0x55ef9c53c380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef9c4b98f0 0x55ef9c4d2850 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9ab400700 0 client.24988 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ef9c4b3a80 0x55ef9c4d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 --2- 172.21.15.90:0/108249108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ef9c4b8fd0 0x55ef9c4d7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 >> 172.21.15.90:0/108249108 conn(0x55ef9c3e1b90 msgr2=0x55ef9c4bc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 shutdown_connections 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 1 -- 172.21.15.90:0/108249108 wait complete. 2024-06-02T15:42:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.335+0000 7ff9d682bb80 20 client.24988 trim_cache size 0 max 16384 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24536, 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 236.85550483, 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:42:16.338 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:42:16.339 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:42:16.339 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 259.30602272700003 2024-06-02T15:42:16.339 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:42:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 msgr2=0x7ff9ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 0x7ff9ac0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009680 tx=0x7ff9b000ff40 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 msgr2=0x7ff9c41a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 0x7ff9c41a05f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00efc0 tx=0x7ff9bc00c3a0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 shutdown_connections 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9ac0b2880 0x7ff9ac0b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c410b650 0x7ff9c41a05f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9c4105650 0x7ff9c419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 --2- 172.21.15.90:0/3897589602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9c4106010 0x7ff9c419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 >> 172.21.15.90:0/3897589602 conn(0x7ff9c4101180 msgr2=0x7ff9c4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 shutdown_connections 2024-06-02T15:42:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:16.336+0000 7ff9ca800700 1 -- 172.21.15.90:0/3897589602 wait complete. 2024-06-02T15:42:17.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:17 smithi090 ceph-mon[31595]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:17.277 INFO:tasks.cephadm:Setup complete, yielding 2024-06-02T15:42:17.277 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-02T15:42:17.288 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:42:17.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-02T15:42:17.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:17 smithi120 ceph-mon[36451]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:17.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:17 smithi135 ceph-mon[36839]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:17.739 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:19.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:19 smithi120 ceph-mon[36451]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-06-02T15:42:19.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:19 smithi135 ceph-mon[36839]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-06-02T15:42:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:19 smithi090 ceph-mon[31595]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-06-02T15:42:20.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 -- 172.21.15.90:0/3685990644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 msgr2=0x7f2700106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:20.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 --2- 172.21.15.90:0/3685990644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f2700106e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0099c0 tx=0x7f26ec009cd0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 -- 172.21.15.90:0/3685990644 shutdown_connections 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 --2- 172.21.15.90:0/3685990644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f270010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 --2- 172.21.15.90:0/3685990644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f2700105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 --2- 172.21.15.90:0/3685990644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f2700106e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 -- 172.21.15.90:0/3685990644 >> 172.21.15.90:0/3685990644 conn(0x7f2700100d90 msgr2=0x7f27001031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 -- 172.21.15.90:0/3685990644 shutdown_connections 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.240+0000 7f2704a00700 1 -- 172.21.15.90:0/3685990644 wait complete. 2024-06-02T15:42:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.241+0000 7f2704a00700 1 Processor -- start 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 -- start start 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f270019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f27001a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27001112b0 con 0x7f27001069f0 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2700111130 con 0x7f27001057f0 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f2704a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2700111430 con 0x7f2700107390 2024-06-02T15:42:20.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f26fd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f26fde00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f270019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f26fd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44190/0 (socket says 172.21.15.90:44190) 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f26fd400700 1 -- 172.21.15.90:0/311549911 learned_addr learned my addr 172.21.15.90:0/311549911 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.242+0000 7f26fe800700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f27001a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 -- 172.21.15.90:0/311549911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 msgr2=0x7f27001a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f27001a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 -- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 msgr2=0x7f270019be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 -- 172.21.15.90:0/311549911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27001a3560 con 0x7f27001057f0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fe800700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f27001a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fd400700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.243+0000 7f26fde00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f270019b900 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e9c0 tx=0x7f26f000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.244+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f000cbd0 con 0x7f27001057f0 2024-06-02T15:42:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.244+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27001a3850 con 0x7f27001057f0 2024-06-02T15:42:20.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.244+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f00044d0 con 0x7f27001057f0 2024-06-02T15:42:20.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.244+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0003cc0 con 0x7f27001057f0 2024-06-02T15:42:20.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.244+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a3de0 con 0x7f27001057f0 2024-06-02T15:42:20.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.245+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270006a230 con 0x7f27001057f0 2024-06-02T15:42:20.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.247+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26f0003e60 con 0x7f27001057f0 2024-06-02T15:42:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.249+0000 7f26f6a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 0x7f26e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.249+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f26f00e28b0 con 0x7f27001057f0 2024-06-02T15:42:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.249+0000 7f26fd400700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 0x7f26e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.250+0000 7f26fd400700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 0x7f26e40b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26ec003820 tx=0x7f26ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.251+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26f00aa1c0 con 0x7f27001057f0 2024-06-02T15:42:20.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.508+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2700063980 con 0x7f26e40b2880 2024-06-02T15:42:20.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.509+0000 7f26f6a00700 1 -- 172.21.15.90:0/311549911 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2700063980 con 0x7f26e40b2880 2024-06-02T15:42:20.507 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-06-02T15:42:20.507 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-06-02T15:42:20.507 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-06-02T15:42:20.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.512+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 msgr2=0x7f26e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:20.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.512+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 0x7f26e40b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26ec003820 tx=0x7f26ec005fb0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.512+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 msgr2=0x7f270019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:20.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.512+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f270019b900 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e9c0 tx=0x7f26f000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 shutdown_connections 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f26e40b2880 0x7f26e40b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2700107390 0x7f27001a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27001057f0 0x7f270019b900 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 --2- 172.21.15.90:0/311549911 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27001069f0 0x7f270019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 >> 172.21.15.90:0/311549911 conn(0x7f2700100d90 msgr2=0x7f2700109d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 shutdown_connections 2024-06-02T15:42:20.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:20.513+0000 7f2704a00700 1 -- 172.21.15.90:0/311549911 wait complete. 2024-06-02T15:42:21.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:21 smithi120 ceph-mon[36451]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:21.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:42:21.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:21 smithi135 ceph-mon[36839]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:21.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:42:21.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:21 smithi090 ceph-mon[31595]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:21.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:42:21.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-02T15:42:21.893 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:22.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:22 smithi120 ceph-mon[36451]: from='client.25025 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:22.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:22 smithi135 ceph-mon[36839]: from='client.25025 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:22.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:22 smithi090 ceph-mon[31595]: from='client.25025 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:23 smithi090 ceph-mon[31595]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-02T15:42:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:23 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:42:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:23 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:42:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:23 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:42:23.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:23 smithi120 ceph-mon[36451]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-02T15:42:23.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:23 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:42:23.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:23 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:42:23.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:23 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:42:23.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:23 smithi135 ceph-mon[36839]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-06-02T15:42:23.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:23 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:42:23.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:23 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:42:23.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:23 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 -- 172.21.15.90:0/1385210273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190106890 msgr2=0x7fd190108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 --2- 172.21.15.90:0/1385210273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190106890 0x7fd190108c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009990 tx=0x7fd17c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 -- 172.21.15.90:0/1385210273 shutdown_connections 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 --2- 172.21.15.90:0/1385210273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 0x7fd19010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 --2- 172.21.15.90:0/1385210273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190103f60 0x7fd190106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 --2- 172.21.15.90:0/1385210273 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190106890 0x7fd190108c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 -- 172.21.15.90:0/1385210273 >> 172.21.15.90:0/1385210273 conn(0x7fd1900fd950 msgr2=0x7fd1900ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 -- 172.21.15.90:0/1385210273 shutdown_connections 2024-06-02T15:42:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.069+0000 7fd18f200700 1 -- 172.21.15.90:0/1385210273 wait complete. 2024-06-02T15:42:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.070+0000 7fd18f200700 1 Processor -- start 2024-06-02T15:42:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 -- start start 2024-06-02T15:42:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190106890 0x7fd19019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 0x7fd1901a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:24.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010ef90 con 0x7fd1901091c0 2024-06-02T15:42:24.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010ee10 con 0x7fd190103f60 2024-06-02T15:42:24.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd19010f110 con 0x7fd190106890 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 0x7fd1901a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47632/0 (socket says 172.21.15.90:47632) 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190106890 0x7fd19019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.071+0000 7fd18de00700 1 -- 172.21.15.90:0/224203754 learned_addr learned my addr 172.21.15.90:0/224203754 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 -- 172.21.15.90:0/224203754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190106890 msgr2=0x7fd19019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190106890 0x7fd19019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 -- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 msgr2=0x7fd1901a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 0x7fd1901a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 -- 172.21.15.90:0/224203754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901a3450 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.072+0000 7fd18de00700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd18000e9c0 tx=0x7fd18000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.073+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18000cbd0 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.073+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18000ee50 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.073+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180005580 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.073+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1901a3740 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.073+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a3c80 con 0x7fd190103f60 2024-06-02T15:42:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.074+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19006a230 con 0x7fd190103f60 2024-06-02T15:42:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.076+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd180047960 con 0x7fd190103f60 2024-06-02T15:42:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.077+0000 7fd186a00700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 0x7fd1740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:24.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.078+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd1800e27c0 con 0x7fd190103f60 2024-06-02T15:42:24.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.078+0000 7fd18d400700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 0x7fd1740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:24.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.078+0000 7fd18d400700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 0x7fd1740b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b2e0 tx=0x7fd17c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:24.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.080+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1800aa0d0 con 0x7fd190103f60 2024-06-02T15:42:24.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.327+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd190063980 con 0x7fd1740b2900 2024-06-02T15:42:24.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.366+0000 7fd186a00700 1 -- 172.21.15.90:0/224203754 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fd190063980 con 0x7fd1740b2900 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.a smithi090 running (4m) 2m ago 4m 16.5M - 19.0.0-3882-gb7328fed 57e49aa75a3f 78c8552a10c1 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.b smithi120 running (4m) 2m ago 4m 26.2M - 19.0.0-3882-gb7328fed 57e49aa75a3f 507a63e63308 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.c smithi135 running (4m) 2m ago 4m 13.7M - 19.0.0-3882-gb7328fed 57e49aa75a3f ca383c9ea6a4 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.d smithi090 running (4m) 2m ago 4m 13.0M - 19.0.0-3882-gb7328fed 57e49aa75a3f 2d69f460223a 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.e smithi120 running (4m) 2m ago 4m 18.2M - 19.0.0-3882-gb7328fed 57e49aa75a3f 52c643c502e6 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.f smithi135 running (4m) 2m ago 4m 13.5M - 19.0.0-3882-gb7328fed 57e49aa75a3f e731c24d4dff 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.g smithi090 running (4m) 2m ago 4m 16.2M - 19.0.0-3882-gb7328fed 57e49aa75a3f 52b4d163c921 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.h smithi120 running (4m) 2m ago 4m 17.8M - 19.0.0-3882-gb7328fed 57e49aa75a3f 48371ec6b816 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.i smithi135 running (4m) 2m ago 4m 16.8M - 19.0.0-3882-gb7328fed 57e49aa75a3f c30288308b22 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.j smithi090 running (5m) 2m ago 5m 13.1M - 19.0.0-3882-gb7328fed 57e49aa75a3f ae4380e2e9d0 2024-06-02T15:42:24.365 INFO:teuthology.orchestra.run.smithi090.stdout:mds.k smithi120 running (5m) 2m ago 5m 14.7M - 19.0.0-3882-gb7328fed 57e49aa75a3f 6a1084a59bf4 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mds.l smithi135 running (5m) 2m ago 5m 17.2M - 19.0.0-3882-gb7328fed 57e49aa75a3f 37e4c077cbde 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi090 *:9283,8765 running (13m) 2m ago 13m 532M - 19.0.0-3882-gb7328fed 57e49aa75a3f 42ce219d0214 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi120 *:8443,8765 running (11m) 2m ago 11m 467M - 19.0.0-3882-gb7328fed 57e49aa75a3f 59a61b2195a0 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.z smithi135 *:8443,8765 running (11m) 2m ago 11m 467M - 19.0.0-3882-gb7328fed 57e49aa75a3f 1d303ccb328d 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (13m) 2m ago 13m 50.1M 2048M 19.0.0-3882-gb7328fed 57e49aa75a3f f7de8496ae94 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi120 running (11m) 2m ago 11m 42.8M 2048M 19.0.0-3882-gb7328fed 57e49aa75a3f ba7a62482c94 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi135 running (11m) 2m ago 11m 44.7M 2048M 19.0.0-3882-gb7328fed 57e49aa75a3f 6a140f208065 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 2m ago 10m 52.3M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 6b5b14de6f3a 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi120 running (10m) 2m ago 10m 46.3M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 3415f422c4cb 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi135 running (9m) 2m ago 9m 56.8M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f e9738fe3c28c 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (9m) 2m ago 9m 44.2M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f aa2e5ff46168 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi120 running (8m) 2m ago 8m 46.6M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f e0ae719f760f 2024-06-02T15:42:24.366 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi135 running (8m) 2m ago 8m 47.2M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 04c3808f7c0a 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi090 running (7m) 2m ago 7m 49.0M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f e08dfbe038f1 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi120 running (7m) 2m ago 7m 44.2M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 3dff34e252b9 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 smithi135 running (7m) 2m ago 7m 52.2M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f ed281141d6b4 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 smithi090 running (6m) 2m ago 6m 50.1M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f e07b5d391cc9 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 smithi120 running (5m) 2m ago 5m 52.4M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 7aea45ecf563 2024-06-02T15:42:24.367 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 smithi135 running (5m) 2m ago 5m 41.7M 4272M 19.0.0-3882-gb7328fed 57e49aa75a3f 038c90639233 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 msgr2=0x7fd1740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 0x7fd1740b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b2e0 tx=0x7fd17c005fb0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 msgr2=0x7fd19019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd18000e9c0 tx=0x7fd18000ecd0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 shutdown_connections 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd1740b2900 0x7fd1740b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1901091c0 0x7fd1901a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd190103f60 0x7fd19019b7f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 --2- 172.21.15.90:0/224203754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd190106890 0x7fd19019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:24.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 >> 172.21.15.90:0/224203754 conn(0x7fd1900fd950 msgr2=0x7fd190103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:24.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.370+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 shutdown_connections 2024-06-02T15:42:24.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:24.371+0000 7fd18f200700 1 -- 172.21.15.90:0/224203754 wait complete. 2024-06-02T15:42:25.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-02T15:42:25.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:25 smithi135 ceph-mon[36839]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:25.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:25 smithi135 ceph-mon[36839]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:25.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:25 smithi090 ceph-mon[31595]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:25.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:25 smithi090 ceph-mon[31595]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:25 smithi120 ceph-mon[36451]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:25 smithi120 ceph-mon[36451]: from='client.25031 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:25.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:27.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:27 smithi135 ceph-mon[36839]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:27.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:27 smithi090 ceph-mon[31595]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:27 smithi120 ceph-mon[36451]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 -- 172.21.15.90:0/1537126874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f281091b0 msgr2=0x7f3f2810b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 --2- 172.21.15.90:0/1537126874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f281091b0 0x7f3f2810b620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b0a0 tx=0x7f3f2400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 -- 172.21.15.90:0/1537126874 shutdown_connections 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 --2- 172.21.15.90:0/1537126874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f281091b0 0x7f3f2810b620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 --2- 172.21.15.90:0/1537126874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f28103f50 0x7f3f28106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 --2- 172.21.15.90:0/1537126874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 0x7f3f28108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 -- 172.21.15.90:0/1537126874 >> 172.21.15.90:0/1537126874 conn(0x7f3f280fd960 msgr2=0x7f3f280ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 -- 172.21.15.90:0/1537126874 shutdown_connections 2024-06-02T15:42:28.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.058+0000 7f3f2f200700 1 -- 172.21.15.90:0/1537126874 wait complete. 2024-06-02T15:42:28.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.059+0000 7f3f2f200700 1 Processor -- start 2024-06-02T15:42:28.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 -- start start 2024-06-02T15:42:28.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:28.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 0x7f3f28101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:28.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 0x7f3f28102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:28.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810ef80 con 0x7f3f28103f50 2024-06-02T15:42:28.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810ee00 con 0x7f3f28106880 2024-06-02T15:42:28.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.060+0000 7f3f2f200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2810f100 con 0x7f3f281091b0 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34472/0 (socket says 172.21.15.90:34472) 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 -- 172.21.15.90:0/2850486101 learned_addr learned my addr 172.21.15.90:0/2850486101 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2e800700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 0x7f3f28102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2d400700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 0x7f3f28101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 -- 172.21.15.90:0/2850486101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 msgr2=0x7f3f28102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 0x7f3f28102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 -- 172.21.15.90:0/2850486101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 msgr2=0x7f3f28101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 0x7f3f28101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2de00700 1 -- 172.21.15.90:0/2850486101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f28102cf0 con 0x7f3f28103f50 2024-06-02T15:42:28.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.061+0000 7f3f2e800700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 0x7f3f28102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f2de00700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00e9c0 tx=0x7f3f1c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c00cbd0 con 0x7f3f28103f50 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f281a36d0 con 0x7f3f28103f50 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1c00cd70 con 0x7f3f28103f50 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c005610 con 0x7f3f28103f50 2024-06-02T15:42:28.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.062+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f281a3be0 con 0x7f3f28103f50 2024-06-02T15:42:28.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.063+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2806a230 con 0x7f3f28103f50 2024-06-02T15:42:28.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.067+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f1c004750 con 0x7f3f28103f50 2024-06-02T15:42:28.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.069+0000 7f3f22a00700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 0x7f3f100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:28.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.069+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3f1c0e2b30 con 0x7f3f28103f50 2024-06-02T15:42:28.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.069+0000 7f3f2d400700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 0x7f3f100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:28.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.070+0000 7f3f2d400700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 0x7f3f100b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007be0 tx=0x7f3f180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:28.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.072+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f1c054df0 con 0x7f3f28103f50 2024-06-02T15:42:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.316+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3f2802e980 con 0x7f3f100b2880 2024-06-02T15:42:28.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.323+0000 7f3f22a00700 1 -- 172.21.15.90:0/2850486101 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3f2802e980 con 0x7f3f100b2880 2024-06-02T15:42:28.321 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-02T15:42:28.321 INFO:teuthology.orchestra.run.smithi090.stdout:mds.all 12/12 2m ago 5m smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi120=b;smithi120=e;smithi120=h;smithi120=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-06-02T15:42:28.321 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 3/3 2m ago 11m smithi090=x;smithi120=y;smithi135=z;count:3 2024-06-02T15:42:28.321 INFO:teuthology.orchestra.run.smithi090.stdout:mon 3/3 2m ago 11m smithi090:172.21.15.90=a;smithi120:172.21.15.120=b;smithi135:172.21.15.135=c;count:3 2024-06-02T15:42:28.321 INFO:teuthology.orchestra.run.smithi090.stdout:osd 12 2m ago - 2024-06-02T15:42:28.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 msgr2=0x7f3f100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:28.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 0x7f3f100b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007be0 tx=0x7f3f180073d0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 msgr2=0x7f3f281038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:28.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00e9c0 tx=0x7f3f1c00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 shutdown_connections 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3f100b2880 0x7f3f100b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f281091b0 0x7f3f28102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f28103f50 0x7f3f281038b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 --2- 172.21.15.90:0/2850486101 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3f28106880 0x7f3f28101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 >> 172.21.15.90:0/2850486101 conn(0x7f3f280fd960 msgr2=0x7f3f28104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 shutdown_connections 2024-06-02T15:42:28.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:28.327+0000 7f3f2f200700 1 -- 172.21.15.90:0/2850486101 wait complete. 2024-06-02T15:42:29.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-02T15:42:29.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:29 smithi090 ceph-mon[31595]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:29.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:29 smithi090 ceph-mon[31595]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:29 smithi120 ceph-mon[36451]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:29 smithi120 ceph-mon[36451]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:29 smithi135 ceph-mon[36839]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-06-02T15:42:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:29 smithi135 ceph-mon[36839]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:29.595 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:31.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:31 smithi090 ceph-mon[31595]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-02T15:42:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:31 smithi120 ceph-mon[36451]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-02T15:42:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:31 smithi135 ceph-mon[36839]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-06-02T15:42:31.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 -- 172.21.15.90:0/2401142088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c1047b0 msgr2=0x7f3d8c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:31.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 --2- 172.21.15.90:0/2401142088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c104c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d780099c0 tx=0x7f3d78009cd0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 -- 172.21.15.90:0/2401142088 shutdown_connections 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 --2- 172.21.15.90:0/2401142088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c105150 0x7f3d8c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 --2- 172.21.15.90:0/2401142088 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 --2- 172.21.15.90:0/2401142088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c104c10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 -- 172.21.15.90:0/2401142088 >> 172.21.15.90:0/2401142088 conn(0x7f3d8c0feb10 msgr2=0x7f3d8c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 -- 172.21.15.90:0/2401142088 shutdown_connections 2024-06-02T15:42:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.646+0000 7f3d92800700 1 -- 172.21.15.90:0/2401142088 wait complete. 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 Processor -- start 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 -- start start 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c105150 0x7f3d8c1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c10f050 con 0x7f3d8c105150 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c10eed0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.647+0000 7f3d92800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c10f1d0 con 0x7f3d8c1047b0 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c105150 0x7f3d8c1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58410/0 (socket says 172.21.15.90:58410) 2024-06-02T15:42:31.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 -- 172.21.15.90:0/3758220021 learned_addr learned my addr 172.21.15.90:0/3758220021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 -- 172.21.15.90:0/3758220021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c1047b0 msgr2=0x7f3d8c19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 -- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c105150 msgr2=0x7f3d8c1a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c105150 0x7f3d8c1a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.648+0000 7f3d8b400700 1 -- 172.21.15.90:0/3758220021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c1a3530 con 0x7f3d8c1035b0 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d8b400700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e970 tx=0x7f3d7c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c004220 con 0x7f3d8c1035b0 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c1a3820 con 0x7f3d8c1035b0 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7c0043c0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c003ce0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.649+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c1a3db0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.650+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c06a230 con 0x7f3d8c1035b0 2024-06-02T15:42:31.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.652+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d7c00c7e0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.654+0000 7f3d83e00700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 0x7f3d700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:31.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.654+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3d7c0e28c0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.654+0000 7f3d8aa00700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 0x7f3d700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:31.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.655+0000 7f3d8aa00700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 0x7f3d700b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:31.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.658+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d7c0aa1d0 con 0x7f3d8c1035b0 2024-06-02T15:42:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.894+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3d8c076e90 con 0x7f3d700b2880 2024-06-02T15:42:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.898+0000 7f3d83e00700 1 -- 172.21.15.90:0/3758220021 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3d8c076e90 con 0x7f3d700b2880 2024-06-02T15:42:31.896 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-06-02T15:42:31.896 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-06-02T15:42:31.896 INFO:teuthology.orchestra.run.smithi090.stdout:smithi120 172.21.15.120 2024-06-02T15:42:31.896 INFO:teuthology.orchestra.run.smithi090.stdout:smithi135 172.21.15.135 2024-06-02T15:42:31.897 INFO:teuthology.orchestra.run.smithi090.stdout:3 hosts in cluster 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 msgr2=0x7f3d700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 0x7f3d700b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c00 tx=0x7f3d78005fb0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 msgr2=0x7f3d8c19b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00e970 tx=0x7f3d7c00ec80 comp rx=0 tx=0).stop 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 shutdown_connections 2024-06-02T15:42:31.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3d700b2880 0x7f3d700b4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d8c105150 0x7f3d8c1a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c1035b0 0x7f3d8c19b8d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 --2- 172.21.15.90:0/3758220021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d8c1047b0 0x7f3d8c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 >> 172.21.15.90:0/3758220021 conn(0x7f3d8c0feb10 msgr2=0x7f3d8c075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 shutdown_connections 2024-06-02T15:42:31.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:31.901+0000 7f3d92800700 1 -- 172.21.15.90:0/3758220021 wait complete. 2024-06-02T15:42:32.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-02T15:42:33.321 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:33 smithi090 ceph-mon[31595]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:33 smithi090 ceph-mon[31595]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:33 smithi120 ceph-mon[36451]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:33 smithi120 ceph-mon[36451]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:33 smithi135 ceph-mon[36839]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:33 smithi135 ceph-mon[36839]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:35.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:35 smithi090 ceph-mon[31595]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 -- 172.21.15.90:0/2790816418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081057e0 msgr2=0x7fcb08105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 --2- 172.21.15.90:0/2790816418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081057e0 0x7fcb08105c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80099c0 tx=0x7fcaf8009cd0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 -- 172.21.15.90:0/2790816418 shutdown_connections 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 --2- 172.21.15.90:0/2790816418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 0x7fcb0810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 --2- 172.21.15.90:0/2790816418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081057e0 0x7fcb08105c00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 --2- 172.21.15.90:0/2790816418 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081069e0 0x7fcb08106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 -- 172.21.15.90:0/2790816418 >> 172.21.15.90:0/2790816418 conn(0x7fcb08100da0 msgr2=0x7fcb081031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:35.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 -- 172.21.15.90:0/2790816418 shutdown_connections 2024-06-02T15:42:35.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.540+0000 7fcb0d200700 1 -- 172.21.15.90:0/2790816418 wait complete. 2024-06-02T15:42:35.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.541+0000 7fcb0d200700 1 Processor -- start 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.541+0000 7fcb0d200700 1 -- start start 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081069e0 0x7fcb0819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 0x7fcb081a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb081112a0 con 0x7fcb081069e0 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08111120 con 0x7fcb081057e0 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb0d200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08111420 con 0x7fcb08107380 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb05e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb05e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58422/0 (socket says 172.21.15.90:58422) 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb05e00700 1 -- 172.21.15.90:0/330766834 learned_addr learned my addr 172.21.15.90:0/330766834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:35.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb05400700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081069e0 0x7fcb0819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.542+0000 7fcb06800700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 0x7fcb081a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 -- 172.21.15.90:0/330766834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 msgr2=0x7fcb081a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 0x7fcb081a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 -- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081069e0 msgr2=0x7fcb0819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081069e0 0x7fcb0819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 -- 172.21.15.90:0/330766834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb081a3580 con 0x7fcb081057e0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb05e00700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009990 tx=0x7fcaf800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8009e60 con 0x7fcb081057e0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf80038c0 con 0x7fcb081057e0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb081a3810 con 0x7fcb081057e0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8005600 con 0x7fcb081057e0 2024-06-02T15:42:35.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.543+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb081a3c60 con 0x7fcb081057e0 2024-06-02T15:42:35.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.544+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0806a230 con 0x7fcb081057e0 2024-06-02T15:42:35.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.547+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcaf800fad0 con 0x7fcb081057e0 2024-06-02T15:42:35.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.549+0000 7fcafea00700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 0x7fcaec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:35.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.549+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcaf80e3720 con 0x7fcb081057e0 2024-06-02T15:42:35.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.549+0000 7fcb05400700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 0x7fcaec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:35.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.550+0000 7fcb05400700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 0x7fcaec0b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009c60 tx=0x7fcaf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:35.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.551+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcaf8013070 con 0x7fcb081057e0 2024-06-02T15:42:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:35 smithi120 ceph-mon[36451]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:35 smithi135 ceph-mon[36839]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.791+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fcb08063980 con 0x7fcaec0b2880 2024-06-02T15:42:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.796+0000 7fcafea00700 1 -- 172.21.15.90:0/330766834 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fcb08063980 con 0x7fcaec0b2880 2024-06-02T15:42:35.794 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-02T15:42:35.794 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-02T15:42:35.794 INFO:teuthology.orchestra.run.smithi090.stdout:smithi120 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XV400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-06-02T15:42:35.794 INFO:teuthology.orchestra.run.smithi090.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-02T15:42:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 msgr2=0x7fcaec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 0x7fcaec0b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009c60 tx=0x7fcaf4009260 comp rx=0 tx=0).stop 2024-06-02T15:42:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 msgr2=0x7fcb0819b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8009990 tx=0x7fcaf800f690 comp rx=0 tx=0).stop 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 shutdown_connections 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcaec0b2880 0x7fcaec0b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb08107380 0x7fcb081a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb081057e0 0x7fcb0819b920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 --2- 172.21.15.90:0/330766834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb081069e0 0x7fcb0819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.799+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 >> 172.21.15.90:0/330766834 conn(0x7fcb08100da0 msgr2=0x7fcb08109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:35.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.800+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 shutdown_connections 2024-06-02T15:42:35.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:35.800+0000 7fcb0d200700 1 -- 172.21.15.90:0/330766834 wait complete. 2024-06-02T15:42:36.768 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-02T15:42:36.778 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:42:36.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-02T15:42:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:37.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:37 smithi090 ceph-mon[31595]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:37.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:37 smithi090 ceph-mon[31595]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:37 smithi120 ceph-mon[36451]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:37.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:37 smithi120 ceph-mon[36451]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:37 smithi135 ceph-mon[36839]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T15:42:37.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:37 smithi135 ceph-mon[36839]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:39.260 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:39 smithi090 ceph-mon[31595]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 -- 172.21.15.90:0/3799919096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe4105060 msgr2=0x7f7fe410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 --2- 172.21.15.90:0/3799919096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe4105060 0x7f7fe410b910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00b0d0 tx=0x7f7fdc00b3e0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 -- 172.21.15.90:0/3799919096 shutdown_connections 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 --2- 172.21.15.90:0/3799919096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe4105060 0x7f7fe410b910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 --2- 172.21.15.90:0/3799919096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41034c0 0x7f7fe41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 --2- 172.21.15.90:0/3799919096 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe41046c0 0x7f7fe4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.431+0000 7f7fe9600700 1 -- 172.21.15.90:0/3799919096 >> 172.21.15.90:0/3799919096 conn(0x7f7fe40fea80 msgr2=0x7f7fe4100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.432+0000 7f7fe9600700 1 -- 172.21.15.90:0/3799919096 shutdown_connections 2024-06-02T15:42:39.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.432+0000 7f7fe9600700 1 -- 172.21.15.90:0/3799919096 wait complete. 2024-06-02T15:42:39.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 Processor -- start 2024-06-02T15:42:39.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 -- start start 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41046c0 0x7f7fe419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 0x7f7fe41a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410ef40 con 0x7f7fe41034c0 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410edc0 con 0x7f7fe4105060 2024-06-02T15:42:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe9600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe410f0c0 con 0x7f7fe41046c0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56960/0 (socket says 172.21.15.90:56960) 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe3e00700 1 -- 172.21.15.90:0/1200233215 learned_addr learned my addr 172.21.15.90:0/1200233215 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.433+0000 7f7fe3400700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41046c0 0x7f7fe419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe8c00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 0x7f7fe41a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 -- 172.21.15.90:0/1200233215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41046c0 msgr2=0x7f7fe419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41046c0 0x7f7fe419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 -- 172.21.15.90:0/1200233215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 msgr2=0x7f7fe41a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 0x7f7fe41a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 -- 172.21.15.90:0/1200233215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41a3390 con 0x7f7fe41034c0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe8c00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 0x7f7fe41a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.434+0000 7f7fe3e00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400d9b0 tx=0x7f7fd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.435+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4016070 con 0x7f7fe41034c0 2024-06-02T15:42:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.435+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41a3680 con 0x7f7fe41034c0 2024-06-02T15:42:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.435+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd400be00 con 0x7f7fe41034c0 2024-06-02T15:42:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.435+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4010e30 con 0x7f7fe41034c0 2024-06-02T15:42:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.435+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe41a3bc0 con 0x7f7fe41034c0 2024-06-02T15:42:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.436+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe406a230 con 0x7f7fe41034c0 2024-06-02T15:42:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.438+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7fd4019070 con 0x7f7fe41034c0 2024-06-02T15:42:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.439+0000 7f7fe0c00700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 0x7f7fc80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.440+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7fd40e2190 con 0x7f7fe41034c0 2024-06-02T15:42:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.440+0000 7f7fe3400700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 0x7f7fc80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.440+0000 7f7fe3400700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 0x7f7fc80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009be0 tx=0x7f7fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:39.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.444+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fd4020030 con 0x7f7fe41034c0 2024-06-02T15:42:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:39 smithi120 ceph-mon[36451]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:39 smithi135 ceph-mon[36839]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:39.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.735+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7fe41a44b0 con 0x7f7fe41034c0 2024-06-02T15:42:39.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.736+0000 7f7fe0c00700 1 -- 172.21.15.90:0/1200233215 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3056 (secure 0 0 0) 0x7f7fd40a9aa0 con 0x7f7fe41034c0 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:e25 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:39.735 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 25 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-06-02T15:38:19.378346+0000 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-06-02T15:39:40.241359+0000 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 300 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-06-02T15:42:39.736 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 3 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1,2 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24536,1=24524,2=24512} 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3,4] 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-06-02T15:42:39.737 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 24536 members: 24536,24512,24524 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b{0:24536} state up:active seq 7 addr [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.h{0:24497} state up:standby-replay seq 1 addr [v2:172.21.15.120:6836/1676077338,v1:172.21.15.120:6837/1676077338] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a{1:24524} state up:active seq 26 addr [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.i{1:24475} state up:standby-replay seq 1 addr [v2:172.21.15.135:6838/3736093015,v1:172.21.15.135:6839/3736093015] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.g{2:24512} state up:active seq 27 addr [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e{2:24473} state up:standby-replay seq 1 addr [v2:172.21.15.120:6834/3465448684,v1:172.21.15.120:6835/3465448684] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.j{-1:14364} state up:standby seq 1 addr [v2:172.21.15.90:6834/661907319,v1:172.21.15.90:6835/661907319] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.738 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.135:6832/3349102106,v1:172.21.15.135:6833/3349102106] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.120:6832/3322691613,v1:172.21.15.120:6833/3322691613] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f{-1:24454} state up:standby seq 1 addr [v2:172.21.15.135:6834/2238315985,v1:172.21.15.135:6835/2238315985] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d{-1:24461} state up:standby seq 1 addr [v2:172.21.15.90:6836/683051134,v1:172.21.15.90:6837/683051134] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.739 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c{-1:24466} state up:standby seq 1 addr [v2:172.21.15.135:6836/284781637,v1:172.21.15.135:6837/284781637] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T15:42:39.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.739+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 msgr2=0x7f7fc80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.739+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 0x7f7fc80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009be0 tx=0x7f7fd0009260 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.739+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 msgr2=0x7f7fe419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.739+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400d9b0 tx=0x7f7fd400dcc0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 shutdown_connections 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7fc80b2880 0x7f7fc80b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7fe4105060 0x7f7fe41a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe41034c0 0x7f7fe419b730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 --2- 172.21.15.90:0/1200233215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fe41046c0 0x7f7fe419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 >> 172.21.15.90:0/1200233215 conn(0x7f7fe40fea80 msgr2=0x7f7fe4075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 shutdown_connections 2024-06-02T15:42:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:39.740+0000 7f7fe9600700 1 -- 172.21.15.90:0/1200233215 wait complete. 2024-06-02T15:42:39.742 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:42:40.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1200233215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T15:42:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1200233215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T15:42:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1200233215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T15:42:40.575 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-02T15:42:41.035 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:41.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:41 smithi090 ceph-mon[31595]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:41.468 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:41 smithi135 ceph-mon[36839]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:41 smithi120 ceph-mon[36451]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:43.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1799262594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc81056e0 msgr2=0x7f9cc8105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:43.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1799262594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc81056e0 0x7f9cc8105b00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009920 tx=0x7f9cb8009c30 comp rx=0 tx=0).stop 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1799262594 shutdown_connections 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1799262594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc8107280 0x7f9cc810db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1799262594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc81056e0 0x7f9cc8105b00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.339+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1799262594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc8106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.340+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1799262594 >> 172.21.15.90:0/1799262594 conn(0x7f9cc8100c60 msgr2=0x7f9cc81030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.340+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1799262594 shutdown_connections 2024-06-02T15:42:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.340+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1799262594 wait complete. 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 Processor -- start 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 -- start start 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 0x7f9cc819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc8107280 0x7f9cc81a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8111150 con 0x7f9cc81068e0 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8110fd0 con 0x7f9cc8107280 2024-06-02T15:42:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc7e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc81112d0 con 0x7f9cc81056e0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.341+0000 7f9cc6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41248/0 (socket says 172.21.15.90:41248) 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 -- 172.21.15.90:0/1163142896 learned_addr learned my addr 172.21.15.90:0/1163142896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc7400700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc8107280 0x7f9cc81a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6a00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 0x7f9cc819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 -- 172.21.15.90:0/1163142896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 msgr2=0x7f9cc819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 0x7f9cc819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 -- 172.21.15.90:0/1163142896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc8107280 msgr2=0x7f9cc81a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc8107280 0x7f9cc81a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 -- 172.21.15.90:0/1163142896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc81a3460 con 0x7f9cc81068e0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6a00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 0x7f9cc819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.342+0000 7f9cc6000700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400e9d0 tx=0x7f9cb400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.343+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb400cbd0 con 0x7f9cc81068e0 2024-06-02T15:42:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.343+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb400ee70 con 0x7f9cc81068e0 2024-06-02T15:42:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.343+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc81a3750 con 0x7f9cc81068e0 2024-06-02T15:42:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.343+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4018680 con 0x7f9cc81068e0 2024-06-02T15:42:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.343+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81a3ce0 con 0x7f9cc81068e0 2024-06-02T15:42:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.344+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc806a230 con 0x7f9cc81068e0 2024-06-02T15:42:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.348+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9cb4010430 con 0x7f9cc81068e0 2024-06-02T15:42:43.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.349+0000 7f9ccca00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 0x7f9cac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:43.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.350+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f9cb40e2aa0 con 0x7f9cc81068e0 2024-06-02T15:42:43.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.350+0000 7f9cc6a00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 0x7f9cac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:43.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.351+0000 7f9cc6a00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 0x7f9cac0b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098f0 tx=0x7f9cb8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:43.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.352+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9cb40aa3b0 con 0x7f9cc81068e0 2024-06-02T15:42:43.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:43 smithi090 ceph-mon[31595]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:43 smithi120 ceph-mon[36451]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:43 smithi135 ceph-mon[36839]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:43.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.587+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f9cc802e560 con 0x7f9cc81068e0 2024-06-02T15:42:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.589+0000 7f9ccca00700 1 -- 172.21.15.90:0/1163142896 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5683 (secure 0 0 0) 0x7f9cb40a9cb0 con 0x7f9cc81068e0 2024-06-02T15:42:43.589 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 90 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-06-02T15:28:38.117986+0000 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-06-02T15:40:00.011486+0000 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 27 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client luminous 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-06-02T15:42:43.590 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 12 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6802/445134562,v1:172.21.15.90:6803/445134562] [v2:172.21.15.90:6804/445134562,v1:172.21.15.90:6805/445134562] exists,up 2ad1d96f-8599-4ee1-9fd1-789f0be13368 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/3632063452,v1:172.21.15.120:6801/3632063452] [v2:172.21.15.120:6802/3632063452,v1:172.21.15.120:6803/3632063452] exists,up 26a9dcac-5eab-4fac-90fa-534adccbf8c7 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/3984900996,v1:172.21.15.135:6801/3984900996] [v2:172.21.15.135:6802/3984900996,v1:172.21.15.135:6803/3984900996] exists,up 8a650ebd-c0d3-4787-bf26-59e8c5ed88f8 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6810/3752831765,v1:172.21.15.90:6811/3752831765] [v2:172.21.15.90:6812/3752831765,v1:172.21.15.90:6813/3752831765] exists,up 276c9e04-fa0c-4005-aef6-72afffc49bab 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3009183394,v1:172.21.15.120:6809/3009183394] [v2:172.21.15.120:6810/3009183394,v1:172.21.15.120:6811/3009183394] exists,up 62594a89-8539-441e-9ae7-fa5938668b30 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/3338529277,v1:172.21.15.135:6809/3338529277] [v2:172.21.15.135:6810/3338529277,v1:172.21.15.135:6811/3338529277] exists,up fd04176c-ed63-4796-99b1-720481048d12 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6818/3483502077,v1:172.21.15.90:6819/3483502077] [v2:172.21.15.90:6820/3483502077,v1:172.21.15.90:6821/3483502077] exists,up f4ffd454-114d-46e2-9735-49849a2fa18b 2024-06-02T15:42:43.591 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 up in weight 1 up_from 48 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6816/1944245064,v1:172.21.15.120:6817/1944245064] [v2:172.21.15.120:6818/1944245064,v1:172.21.15.120:6819/1944245064] exists,up a44884e0-5c29-4809-8204-7e00dcbf3285 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 up in weight 1 up_from 54 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/1073792121,v1:172.21.15.135:6817/1073792121] [v2:172.21.15.135:6818/1073792121,v1:172.21.15.135:6819/1073792121] exists,up 03d2ed37-a237-4c17-a69e-ce75f99dc563 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6826/3920640313,v1:172.21.15.90:6827/3920640313] [v2:172.21.15.90:6828/3920640313,v1:172.21.15.90:6829/3920640313] exists,up a0d14146-6d2a-4b23-bbd2-318bf2dad564 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/1810652587,v1:172.21.15.120:6825/1810652587] [v2:172.21.15.120:6826/1810652587,v1:172.21.15.120:6827/1810652587] exists,up a96be6b1-b9f1-4e56-8394-092362a013ca 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 up in weight 1 up_from 71 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/359143967,v1:172.21.15.135:6825/359143967] [v2:172.21.15.135:6826/359143967,v1:172.21.15.135:6827/359143967] exists,up 7fc0220b-d39c-4245-8784-8334b9d06f67 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.0 [8,4] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.a [2,11] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.14 [2,4] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.1f [7,11] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.6 [4,6] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.2c [4,9] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.2d [3,9] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.36 [4,9] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.37 [5,8] 2024-06-02T15:42:43.592 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.5 [5,8] 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.b [9,8] 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.18 [9,10] 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.19 [9,8] 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2134019795 expires 2024-06-03T15:29:57.531987+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3164241316 expires 2024-06-03T15:29:57.531987+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/1560565398 expires 2024-06-03T15:29:57.531987+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/232748990 expires 2024-06-03T15:29:23.308098+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/828520669 expires 2024-06-03T15:29:23.308098+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1506181662 expires 2024-06-03T15:29:23.308098+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/1560565398 expires 2024-06-03T15:29:57.531987+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/4260623570 expires 2024-06-03T15:29:23.308098+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/4260623570 expires 2024-06-03T15:29:23.308098+0000 2024-06-02T15:42:43.593 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1946492510 expires 2024-06-03T15:29:57.531987+0000 2024-06-02T15:42:43.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.592+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 msgr2=0x7f9cac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:43.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.592+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 0x7f9cac0b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098f0 tx=0x7f9cb8019040 comp rx=0 tx=0).stop 2024-06-02T15:42:43.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.592+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 msgr2=0x7f9cc819bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:43.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.592+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400e9d0 tx=0x7f9cb400ece0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 shutdown_connections 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9cac0b2900 0x7f9cac0b4dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cc8107280 0x7f9cc81a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cc81056e0 0x7f9cc819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 --2- 172.21.15.90:0/1163142896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9cc81068e0 0x7f9cc819bd90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 >> 172.21.15.90:0/1163142896 conn(0x7f9cc8100c60 msgr2=0x7f9cc8109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 shutdown_connections 2024-06-02T15:42:43.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:43.593+0000 7f9cc7e00700 1 -- 172.21.15.90:0/1163142896 wait complete. 2024-06-02T15:42:44.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:44 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1163142896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T15:42:44.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:44 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1163142896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T15:42:44.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:44 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1163142896' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T15:42:44.619 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-02T15:42:44.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:42:45.081 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:45.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:45 smithi090 ceph-mon[31595]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:45 smithi120 ceph-mon[36451]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:45 smithi135 ceph-mon[36839]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.380+0000 7fb720a00700 1 -- 172.21.15.90:0/239611997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c10b650 msgr2=0x7fb71c10ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.380+0000 7fb720a00700 1 --2- 172.21.15.90:0/239611997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c10b650 0x7fb71c10ba30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009990 tx=0x7fb70c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 -- 172.21.15.90:0/239611997 shutdown_connections 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 --2- 172.21.15.90:0/239611997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 --2- 172.21.15.90:0/239611997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c10b650 0x7fb71c10ba30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 --2- 172.21.15.90:0/239611997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c105650 0x7fb71c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 -- 172.21.15.90:0/239611997 >> 172.21.15.90:0/239611997 conn(0x7fb71c101120 msgr2=0x7fb71c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:47.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 -- 172.21.15.90:0/239611997 shutdown_connections 2024-06-02T15:42:47.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.381+0000 7fb720a00700 1 -- 172.21.15.90:0/239611997 wait complete. 2024-06-02T15:42:47.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 Processor -- start 2024-06-02T15:42:47.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 -- start start 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 0x7fb71c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1158a0 con 0x7fb71c105650 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c115720 con 0x7fb71c106010 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.382+0000 7fb720a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c115a20 con 0x7fb71c10b650 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:47.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41274/0 (socket says 172.21.15.90:41274) 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 -- 172.21.15.90:0/3615867880 learned_addr learned my addr 172.21.15.90:0/3615867880 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71be00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 0x7fb71c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71aa00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 -- 172.21.15.90:0/3615867880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 msgr2=0x7fb71c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 0x7fb71c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 -- 172.21.15.90:0/3615867880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 msgr2=0x7fb71c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71b400700 1 -- 172.21.15.90:0/3615867880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1a0d60 con 0x7fb71c105650 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71aa00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.383+0000 7fb71be00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 0x7fb71c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:47.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:47 smithi090 ceph-mon[31595]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:47.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb71b400700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000c00 tx=0x7fb70c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:47.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c027070 con 0x7fb71c105650 2024-06-02T15:42:47.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb70c022470 con 0x7fb71c105650 2024-06-02T15:42:47.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1a0ff0 con 0x7fb71c105650 2024-06-02T15:42:47.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c017650 con 0x7fb71c105650 2024-06-02T15:42:47.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.384+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a88d0 con 0x7fb71c105650 2024-06-02T15:42:47.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.386+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c06a230 con 0x7fb71c105650 2024-06-02T15:42:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.389+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb70c02a070 con 0x7fb71c105650 2024-06-02T15:42:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.390+0000 7fb713e00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 0x7fb7000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.390+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb70c0e35c0 con 0x7fb71c105650 2024-06-02T15:42:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.390+0000 7fb71aa00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 0x7fb7000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:47.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.391+0000 7fb71aa00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 0x7fb7000b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb71c103740 tx=0x7fb708009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.392+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb70c013070 con 0x7fb71c105650 2024-06-02T15:42:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:47 smithi120 ceph-mon[36451]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:47 smithi135 ceph-mon[36839]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.674+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb71c02e560 con 0x7fb71c105650 2024-06-02T15:42:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.676+0000 7fb713e00700 1 -- 172.21.15.90:0/3615867880 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7fb70c0aaf60 con 0x7fb71c105650 2024-06-02T15:42:47.674 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 msgr2=0x7fb7000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 0x7fb7000b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb71c103740 tx=0x7fb708009260 comp rx=0 tx=0).stop 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 msgr2=0x7fb71c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb70c000c00 tx=0x7fb70c0057c0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 shutdown_connections 2024-06-02T15:42:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7000b2900 0x7fb7000b4dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb71c10b650 0x7fb71c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c105650 0x7fb71c19bd50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 --2- 172.21.15.90:0/3615867880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb71c106010 0x7fb71c19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 >> 172.21.15.90:0/3615867880 conn(0x7fb71c101120 msgr2=0x7fb71c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 shutdown_connections 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:47.679+0000 7fb720a00700 1 -- 172.21.15.90:0/3615867880 wait complete. 2024-06-02T15:42:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:42:48.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:48 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3615867880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:48.552 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:42:48.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:48 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3615867880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:48.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:48 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3615867880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:48.985 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:49.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:49 smithi090 ceph-mon[31595]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:49 smithi120 ceph-mon[36451]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:49 smithi135 ceph-mon[36839]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:51.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:51 smithi090 ceph-mon[31595]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:51.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.394+0000 7f17ca600700 1 -- 172.21.15.90:0/3340122013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 msgr2=0x7f17c4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:51.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.394+0000 7f17ca600700 1 --2- 172.21.15.90:0/3340122013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c4106750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f17b00099c0 tx=0x7f17b0009cd0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 -- 172.21.15.90:0/3340122013 shutdown_connections 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 --2- 172.21.15.90:0/3340122013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 0x7f17c410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 --2- 172.21.15.90:0/3340122013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 --2- 172.21.15.90:0/3340122013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c4106750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 -- 172.21.15.90:0/3340122013 >> 172.21.15.90:0/3340122013 conn(0x7f17c40771e0 msgr2=0x7f17c40775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 -- 172.21.15.90:0/3340122013 shutdown_connections 2024-06-02T15:42:51.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.395+0000 7f17ca600700 1 -- 172.21.15.90:0/3340122013 wait complete. 2024-06-02T15:42:51.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 Processor -- start 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 -- start start 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 0x7f17c4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4111480 con 0x7f17c4102d80 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4111300 con 0x7f17c4106e30 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.396+0000 7f17ca600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4111600 con 0x7f17c41023d0 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48450/0 (socket says 172.21.15.90:48450) 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 -- 172.21.15.90:0/3300192731 learned_addr learned my addr 172.21.15.90:0/3300192731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9c00700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 0x7f17c4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:51.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 -- 172.21.15.90:0/3300192731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 msgr2=0x7f17c4199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:51.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 0x7f17c4199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 -- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 msgr2=0x7f17c419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:51.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.397+0000 7f17c9200700 1 -- 172.21.15.90:0/3300192731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c419a080 con 0x7f17c41023d0 2024-06-02T15:42:51.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17c9200700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9c0 tx=0x7f17b800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b800cbd0 con 0x7f17c41023d0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17b800ee60 con 0x7f17c41023d0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b80055c0 con 0x7f17c41023d0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c419a370 con 0x7f17c41023d0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.398+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c4103720 con 0x7f17c41023d0 2024-06-02T15:42:51.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.399+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c406a230 con 0x7f17c41023d0 2024-06-02T15:42:51.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.402+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17b8004750 con 0x7f17c41023d0 2024-06-02T15:42:51.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.403+0000 7f17c1600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 0x7f17ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:51.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.403+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f17b80e2840 con 0x7f17c41023d0 2024-06-02T15:42:51.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.403+0000 7f17c3e00700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 0x7f17ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:51.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.404+0000 7f17c3e00700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 0x7f17ac0b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:51.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.407+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17b80aa1b0 con 0x7f17c41023d0 2024-06-02T15:42:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:51 smithi120 ceph-mon[36451]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:51 smithi135 ceph-mon[36839]: pgmap v466: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:51.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.688+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17c419ae50 con 0x7f17c41023d0 2024-06-02T15:42:51.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.690+0000 7f17c1600700 1 -- 172.21.15.90:0/3300192731 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11877 (secure 0 0 0) 0x7f17b80a9ab0 con 0x7f17c41023d0 2024-06-02T15:42:51.687 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:51.688 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:39:40.241359+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:42:51.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 msgr2=0x7f17ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 0x7f17ac0b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000c00 tx=0x7f17b0005fb0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 msgr2=0x7f17c419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17b800e9c0 tx=0x7f17b800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 shutdown_connections 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17ac0b2880 0x7f17ac0b4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17c4106e30 0x7f17c4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17c41023d0 0x7f17c419f760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 --2- 172.21.15.90:0/3300192731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17c4102d80 0x7f17c419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.693+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 >> 172.21.15.90:0/3300192731 conn(0x7f17c40771e0 msgr2=0x7f17c41000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.694+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 shutdown_connections 2024-06-02T15:42:51.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:51.694+0000 7f17ca600700 1 -- 172.21.15.90:0/3300192731 wait complete. 2024-06-02T15:42:51.692 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-06-02T15:42:52.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:52 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3300192731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:52.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:52 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3300192731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:52.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:52 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3300192731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:42:52.596 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:42:53.035 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:53.506 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:53 smithi090 ceph-mon[31595]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:53 smithi120 ceph-mon[36451]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:53 smithi135 ceph-mon[36839]: pgmap v467: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:42:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 -- 172.21.15.90:0/2543785439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac06c630 msgr2=0x7f83ac1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 --2- 172.21.15.90:0/2543785439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac06c630 0x7f83ac1103d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009f90 tx=0x7f83a400b330 comp rx=0 tx=0).stop 2024-06-02T15:42:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 -- 172.21.15.90:0/2543785439 shutdown_connections 2024-06-02T15:42:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 --2- 172.21.15.90:0/2543785439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac06c630 0x7f83ac1103d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 --2- 172.21.15.90:0/2543785439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac1095b0 0x7f83ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 --2- 172.21.15.90:0/2543785439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06bc70 0x7f83ac06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 -- 172.21.15.90:0/2543785439 >> 172.21.15.90:0/2543785439 conn(0x7f83ac0793f0 msgr2=0x7f83ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 -- 172.21.15.90:0/2543785439 shutdown_connections 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.407+0000 7f83b1a00700 1 -- 172.21.15.90:0/2543785439 wait complete. 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 Processor -- start 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 -- start start 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 0x7f83ac102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac1095b0 0x7f83ac102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac113730 con 0x7f83ac06c630 2024-06-02T15:42:55.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1135b0 con 0x7f83ac06bc70 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.408+0000 7f83b1a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1138b0 con 0x7f83ac1095b0 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 0x7f83ac102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aaa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aaa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:56142/0 (socket says 172.21.15.90:56142) 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aaa00700 1 -- 172.21.15.90:0/4072808419 learned_addr learned my addr 172.21.15.90:0/4072808419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83ab400700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac1095b0 0x7f83ac102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 -- 172.21.15.90:0/4072808419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac1095b0 msgr2=0x7f83ac102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac1095b0 0x7f83ac102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 -- 172.21.15.90:0/4072808419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 msgr2=0x7f83ac102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 -- 172.21.15.90:0/4072808419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac1a81b0 con 0x7f83ac06c630 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aaa00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.409+0000 7f83aa000700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 0x7f83ac102a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f839800eb90 tx=0x7f839800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:55.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.410+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839800cc90 con 0x7f83ac06c630 2024-06-02T15:42:55.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.410+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839800ce30 con 0x7f83ac06c630 2024-06-02T15:42:55.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.410+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac1a84a0 con 0x7f83ac06c630 2024-06-02T15:42:55.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.410+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398010430 con 0x7f83ac06c630 2024-06-02T15:42:55.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.410+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac1a89e0 con 0x7f83ac06c630 2024-06-02T15:42:55.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.411+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac10ba90 con 0x7f83ac06c630 2024-06-02T15:42:55.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.414+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8398003ef0 con 0x7f83ac06c630 2024-06-02T15:42:55.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.415+0000 7f83a3400700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 0x7f83900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:55.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.416+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f83980e2900 con 0x7f83ac06c630 2024-06-02T15:42:55.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.416+0000 7f83aaa00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 0x7f83900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:55.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.416+0000 7f83aaa00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 0x7f83900b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f839c011550 tx=0x7f839c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:55.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.420+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83980aa210 con 0x7f83ac06c630 2024-06-02T15:42:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:55 smithi120 ceph-mon[36451]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:55 smithi135 ceph-mon[36839]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:55 smithi090 ceph-mon[31595]: pgmap v468: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:55.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.654+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83ac103a70 con 0x7f83ac06c630 2024-06-02T15:42:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.659+0000 7f83a3400700 1 -- 172.21.15.90:0/4072808419 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7f83980a9b10 con 0x7f83ac06c630 2024-06-02T15:42:55.657 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:42:55.657 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:42:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 msgr2=0x7f83900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 0x7f83900b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f839c011550 tx=0x7f839c009260 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 msgr2=0x7f83ac102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 0x7f83ac102a80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f839800eb90 tx=0x7f839800eea0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 shutdown_connections 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83900b2880 0x7f83900b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83ac1095b0 0x7f83ac102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83ac06bc70 0x7f83ac102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 --2- 172.21.15.90:0/4072808419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83ac06c630 0x7f83ac102a80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.662+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 >> 172.21.15.90:0/4072808419 conn(0x7f83ac0793f0 msgr2=0x7f83ac1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.663+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 shutdown_connections 2024-06-02T15:42:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:55.663+0000 7f83b1a00700 1 -- 172.21.15.90:0/4072808419 wait complete. 2024-06-02T15:42:56.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:56 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4072808419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:56.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-02T15:38:19.378346+0000', 'modified': '2024-06-02T15:39:40.241359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24524, 'mds_2': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6835/3465448684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6834', 'nonce': 3465448684}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 3465448684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6839/3736093015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 3736093015}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 3736093015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.120:6837/1676077338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6836', 'nonce': 1676077338}, {'type': 'v1', 'addr': '172.21.15.120:6837', 'nonce': 1676077338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'g', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.90:6839/1414255295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 1414255295}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 1414255295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.90:6841/1722073812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 1722073812}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 1722073812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.120:6839/3642613472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6838', 'nonce': 3642613472}, {'type': 'v1', 'addr': '172.21.15.120:6839', 'nonce': 3642613472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24536, 'qdb_cluster': [24536, 24512, 24524]} 2024-06-02T15:42:56.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-06-02T15:42:56.499 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.g status 2024-06-02T15:42:56.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:56 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4072808419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:56.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:56 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4072808419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:42:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:42:57.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:57 smithi090 ceph-mon[31595]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:57 smithi120 ceph-mon[36451]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:57 smithi135 ceph-mon[36839]: pgmap v469: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:42:59.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:42:59 smithi090 ceph-mon[31595]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3265576942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 msgr2=0x7fd9dc109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3265576942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc109990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0098f0 tx=0x7fd9cc009c00 comp rx=0 tx=0).stop 2024-06-02T15:42:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3265576942 shutdown_connections 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3265576942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3265576942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc109990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3265576942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06b220 0x7fd9dc06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3265576942 >> 172.21.15.90:0/3265576942 conn(0x7fd9dc0793f0 msgr2=0x7fd9dc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3265576942 shutdown_connections 2024-06-02T15:42:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.289+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3265576942 wait complete. 2024-06-02T15:42:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.290+0000 7fd9e1e00700 1 Processor -- start 2024-06-02T15:42:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.290+0000 7fd9e1e00700 1 -- start start 2024-06-02T15:42:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 0x7fd9dc19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc113720 con 0x7fd9dc06b220 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1135a0 con 0x7fd9dc1095b0 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9e1e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc1138a0 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9da000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9da000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48504/0 (socket says 172.21.15.90:48504) 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9da000700 1 -- 172.21.15.90:0/3986863771 learned_addr learned my addr 172.21.15.90:0/3986863771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.291+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 0x7fd9dc19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9db400700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 -- 172.21.15.90:0/3986863771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 msgr2=0x7fd9dc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 -- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 msgr2=0x7fd9dc19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 0x7fd9dc19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 -- 172.21.15.90:0/3986863771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1a0ed0 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 0x7fd9dc19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.292+0000 7fd9da000700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9a0 tx=0x7fd9c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.293+0000 7fd9e0a00700 1 -- 172.21.15.90:0/3986863771 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800cbd0 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.293+0000 7fd9e0a00700 1 -- 172.21.15.90:0/3986863771 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c800cd70 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.293+0000 7fd9e0a00700 1 -- 172.21.15.90:0/3986863771 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8018680 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.293+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc1a11c0 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.293+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1a8a40 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.298+0000 7fd9e0a00700 1 -- 172.21.15.90:0/3986863771 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd9c8010430 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.298+0000 7fd9ec749b80 1 Processor -- start 2024-06-02T15:42:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 -- start start 2024-06-02T15:42:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a759a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555cf69cddf0 con 0x555cf6a755c0 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555cf69bd990 con 0x555cf6a75ee0 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.299+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf69c19d0 con 0x555cf6a700f0 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9da000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9da000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a70570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48516/0 (socket says 172.21.15.90:48516) 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9da000700 1 -- 172.21.15.90:0/3075735230 learned_addr learned my addr 172.21.15.90:0/3075735230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9e0a00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 0x7fd9c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 -- 172.21.15.90:0/3075735230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 msgr2=0x555cf6a70570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a70570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 -- 172.21.15.90:0/3075735230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 msgr2=0x555cf6a759a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a759a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9e0a00700 1 -- 172.21.15.90:0/3986863771 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd9c80e2830 con 0x7fd9dc06bbe0 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9daa00700 1 -- 172.21.15.90:0/3075735230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf69863d0 con 0x555cf6a75ee0 2024-06-02T15:42:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9db400700 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:42:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.300+0000 7fd9db400700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 0x7fd9c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.301+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a6fad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c010 tx=0x7fd9cc00bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de07dcfe3461e03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.301+0000 7fd9d0c00700 1 -- 172.21.15.90:0/3075735230 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc00f720 con 0x555cf6a75ee0 2024-06-02T15:42:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.301+0000 7fd9d0c00700 1 -- 172.21.15.90:0/3075735230 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc00f8c0 con 0x555cf6a75ee0 2024-06-02T15:42:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.301+0000 7fd9d0c00700 1 -- 172.21.15.90:0/3075735230 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004bd0 con 0x555cf6a75ee0 2024-06-02T15:42:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.301+0000 7fd9db400700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 0x7fd9c00b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009570 tx=0x7fd9d4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3075735230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 msgr2=0x555cf6a6fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a6fad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00c010 tx=0x7fd9cc00bf60 comp rx=0 tx=0).stop 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3075735230 shutdown_connections 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a70570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a759a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3075735230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a6fad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3075735230 >> 172.21.15.90:0/3075735230 conn(0x555cf699db90 msgr2=0x555cf6a6a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3075735230 shutdown_connections 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.302+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3075735230 wait complete. 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.303+0000 7fd9ec749b80 1 -- start start 2024-06-02T15:42:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.303+0000 7fd9ec749b80 1 Processor -- start 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a7bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a78fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a79520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555cf68c3f50 con 0x555cf6a755c0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555cf695e710 con 0x555cf6a75ee0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a85140 con 0x555cf6a700f0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9ec749b80 10 client.0 _ll_register_callbacks cb 0x555cf6967c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9da000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a78fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a7bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9da000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a78fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50852/0 (socket says 172.21.15.90:50852) 2024-06-02T15:42:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9da000700 1 -- 172.21.15.90:0/1146532964 learned_addr learned my addr 172.21.15.90:0/1146532964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.304+0000 7fd9db400700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a79520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 -- 172.21.15.90:0/1146532964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 msgr2=0x555cf6a79520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a79520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 -- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 msgr2=0x555cf6a78fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a78fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 -- 172.21.15.90:0/1146532964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf690d880 con 0x555cf6a700f0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9daa00700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a7bc40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0098c0 tx=0x7fd9cc00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672b1a70e10178a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9c5600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004030 con 0x555cf6a700f0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd9cc0041d0 con 0x555cf6a700f0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.305+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9cc00f8c0 con 0x555cf6a700f0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9cc03c050 con 0x555cf6a700f0 2024-06-02T15:42:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 10 client.25060 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:59.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cf698e010 con 0x555cf6a700f0 2024-06-02T15:42:59.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc00f8c0 con 0x555cf6a700f0 2024-06-02T15:42:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9cc02c430 con 0x555cf6a700f0 2024-06-02T15:42:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 10 client.25060 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 10 client.25060 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-02T15:42:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 20 client.25060 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6a91780 0x555cf6a92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 4 client.25060 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.306+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf69cfd40 con 0x555cf6a91780 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.307+0000 7fd9da000700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6a91780 0x555cf6a92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.307+0000 7fd9da000700 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6a91780 0x555cf6a92bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ddef7d3fb45e20b server_cookie=ce19d02648fd4287 in_seq=0 out_seq=0 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.307+0000 7fd9c5600700 10 client.25060 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.308+0000 7fd9c5600700 1 -- 172.21.15.90:0/1146532964 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555cf69cfd40 con 0x555cf6a91780 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.308+0000 7fd9c5600700 10 client.25060 handle_command_reply: tid=0 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.308+0000 7fd9ec749b80 1 client.25060 shutdown 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.308+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 msgr2=0x555cf6a7bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.308+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a7bc40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0098c0 tx=0x7fd9cc00b810 comp rx=0 tx=0).stop 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 shutdown_connections 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6a91780 0x555cf6a92bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a75ee0 0x555cf6a79520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a7bc40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/1146532964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a755c0 0x555cf6a78fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 >> 172.21.15.90:0/1146532964 conn(0x555cf699db90 msgr2=0x555cf6a69d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9c5600700 0 client.25060 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 shutdown_connections 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 1 -- 172.21.15.90:0/1146532964 wait complete. 2024-06-02T15:42:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.309+0000 7fd9ec749b80 20 client.25060 trim_cache size 0 max 16384 2024-06-02T15:42:59.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 Processor -- start 2024-06-02T15:42:59.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 -- start start 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a90f30 con 0x555cf6a75ee0 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a91600 con 0x555cf6a755c0 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.332+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a91cd0 con 0x555cf6a700f0 2024-06-02T15:42:59.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48524/0 (socket says 172.21.15.90:48524) 2024-06-02T15:42:59.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50854/0 (socket says 172.21.15.90:50854) 2024-06-02T15:42:59.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 -- 172.21.15.90:0/722341785 learned_addr learned my addr 172.21.15.90:0/722341785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:59.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9da000700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 -- 172.21.15.90:0/722341785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 msgr2=0x555cf6a704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 -- 172.21.15.90:0/722341785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 msgr2=0x555cf6a769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9daa00700 1 -- 172.21.15.90:0/722341785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf6a91420 con 0x555cf6a700f0 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.333+0000 7fd9da000700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.334+0000 7fd9db400700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.334+0000 7fd9daa00700 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3239dd1d20e07833 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.334+0000 7fd9c7e00700 1 -- 172.21.15.90:0/722341785 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc02c430 con 0x555cf6a700f0 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.334+0000 7fd9c7e00700 1 -- 172.21.15.90:0/722341785 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc02c5d0 con 0x555cf6a700f0 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.334+0000 7fd9c7e00700 1 -- 172.21.15.90:0/722341785 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0047a0 con 0x555cf6a700f0 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 -- 172.21.15.90:0/722341785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 msgr2=0x555cf6a66990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc000c00 tx=0x7fd9cc005bc0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 -- 172.21.15.90:0/722341785 shutdown_connections 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a75ee0 0x555cf6a769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a700f0 0x555cf6a66990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/722341785 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 -- 172.21.15.90:0/722341785 >> 172.21.15.90:0/722341785 conn(0x555cf699db90 msgr2=0x555cf6a762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 -- 172.21.15.90:0/722341785 shutdown_connections 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.335+0000 7fd9ec749b80 1 -- 172.21.15.90:0/722341785 wait complete. 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 -- start start 2024-06-02T15:42:59.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 Processor -- start 2024-06-02T15:42:59.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a700f0 0x555cf6a9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 0x555cf6a69df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a92a70 con 0x555cf6a700f0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a923a0 con 0x555cf6a755c0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf6a93140 con 0x555cf6a75ee0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.336+0000 7fd9ec749b80 10 client.0 _ll_register_callbacks cb 0x555cf6967c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:56212/0 (socket says 172.21.15.90:56212) 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 -- 172.21.15.90:0/3460424900 learned_addr learned my addr 172.21.15.90:0/3460424900 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a700f0 0x555cf6a9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9db400700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 0x555cf6a69df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 -- 172.21.15.90:0/3460424900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 msgr2=0x555cf6a69df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 0x555cf6a69df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 -- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a700f0 msgr2=0x555cf6a9c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:42:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a700f0 0x555cf6a9c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 -- 172.21.15.90:0/3460424900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf6a6a610 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9db400700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 0x555cf6a69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9da000700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80a5ac0 tx=0x7fd9c80a5a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98115260522b98ec server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.337+0000 7fd9c5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c808f060 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd9c8010720 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9cc02c5d0 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9c80b2040 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9ec749b80 10 client.25097 fetch_fsmap learned FSMap version 25 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cf6afcfc0 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc02c5d0 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9c80cc0b0 con 0x555cf6a755c0 2024-06-02T15:42:59.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9ec749b80 10 client.25097 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.338+0000 7fd9ec749b80 10 client.25097 resolve_mds: resolved name 'g' to daemon mds.g 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9ec749b80 20 client.25097 populate_metadata read hostname 'smithi090' 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6b01b30 0x555cf6b00840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9ec749b80 4 client.25097 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "status"}) -- 0x555cf69c9520 con 0x555cf6b01b30 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6b01b30 0x555cf6b00840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9daa00700 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6b01b30 0x555cf6b00840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=533a45d6e09a284 server_cookie=f10c12835afa6de7 in_seq=0 out_seq=0 2024-06-02T15:42:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.339+0000 7fd9c5600700 10 client.25097 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:59.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9c5600700 1 -- 172.21.15.90:0/3460424900 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555cf69c9520 con 0x555cf6b01b30 2024-06-02T15:42:59.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9c5600700 10 client.25097 handle_command_reply: tid=0 2024-06-02T15:42:59.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 client.25097 shutdown 2024-06-02T15:42:59.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 msgr2=0x555cf6a698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80a5ac0 tx=0x7fd9c80a5a30 comp rx=0 tx=0).stop 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 shutdown_connections 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555cf6b01b30 0x555cf6b00840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf6a75ee0 0x555cf6a69df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555cf6a700f0 0x555cf6a9c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9c5600700 0 client.25097 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 --2- 172.21.15.90:0/3460424900 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555cf6a755c0 0x555cf6a698b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 >> 172.21.15.90:0/3460424900 conn(0x555cf699db90 msgr2=0x555cf6a762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 shutdown_connections 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.340+0000 7fd9ec749b80 1 -- 172.21.15.90:0/3460424900 wait complete. 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9ec749b80 20 client.25097 trim_cache size 0 max 16384 2024-06-02T15:42:59.341 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24512, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 206.43687811199999, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 309.909573325 2024-06-02T15:42:59.342 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 msgr2=0x7fd9c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 0x7fd9c00b4dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009570 tx=0x7fd9d4009040 comp rx=0 tx=0).stop 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 msgr2=0x7fd9dc19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9a0 tx=0x7fd9c800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 shutdown_connections 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2900 0x7fd9c00b4dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc1095b0 0x7fd9dc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc06b220 0x7fd9dc19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 --2- 172.21.15.90:0/3986863771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc06bbe0 0x7fd9dc19c400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:42:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.341+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 >> 172.21.15.90:0/3986863771 conn(0x7fd9dc0793f0 msgr2=0x7fd9dc10e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:42:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.342+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 shutdown_connections 2024-06-02T15:42:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:42:59.342+0000 7fd9e1e00700 1 -- 172.21.15.90:0/3986863771 wait complete. 2024-06-02T15:42:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:42:59 smithi120 ceph-mon[36451]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:42:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:42:59 smithi135 ceph-mon[36839]: pgmap v470: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:00.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.a status 2024-06-02T15:43:00.663 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:01 smithi090 ceph-mon[31595]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:01 smithi120 ceph-mon[36451]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:01 smithi135 ceph-mon[36839]: pgmap v471: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 -- 172.21.15.90:0/87528476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca410b660 msgr2=0x7f2ca410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 --2- 172.21.15.90:0/87528476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca410b660 0x7f2ca410ba40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c980099a0 tx=0x7f2c98009cb0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 -- 172.21.15.90:0/87528476 shutdown_connections 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 --2- 172.21.15.90:0/87528476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4106020 0x7f2ca4112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 --2- 172.21.15.90:0/87528476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca410b660 0x7f2ca410ba40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 --2- 172.21.15.90:0/87528476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4105660 0x7f2ca4105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 -- 172.21.15.90:0/87528476 >> 172.21.15.90:0/87528476 conn(0x7f2ca41010f0 msgr2=0x7f2ca4103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 -- 172.21.15.90:0/87528476 shutdown_connections 2024-06-02T15:43:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.922+0000 7f2caa200700 1 -- 172.21.15.90:0/87528476 wait complete. 2024-06-02T15:43:02.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.923+0000 7f2caa200700 1 Processor -- start 2024-06-02T15:43:02.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.923+0000 7f2caa200700 1 -- start start 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4106020 0x7f2ca419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca410b660 0x7f2ca41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41158b0 con 0x7f2ca4106020 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4115730 con 0x7f2ca4105660 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2caa200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4115a30 con 0x7f2ca410b660 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4106020 0x7f2ca419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35216/0 (socket says 172.21.15.90:35216) 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2ca8e00700 1 -- 172.21.15.90:0/1665563198 learned_addr learned my addr 172.21.15.90:0/1665563198 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.924+0000 7f2ca9800700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca410b660 0x7f2ca41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 -- 172.21.15.90:0/1665563198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca410b660 msgr2=0x7f2ca41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca410b660 0x7f2ca41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 -- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4106020 msgr2=0x7f2ca419c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4106020 0x7f2ca419c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 -- 172.21.15.90:0/1665563198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca41a0d70 con 0x7f2ca4105660 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c980094a0 tx=0x7f2c98004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca1600700 1 -- 172.21.15.90:0/1665563198 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98026070 con 0x7f2ca4105660 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca41a1000 con 0x7f2ca4105660 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca1600700 1 -- 172.21.15.90:0/1665563198 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c9800bcd0 con 0x7f2ca4105660 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2ca1600700 1 -- 172.21.15.90:0/1665563198 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98013070 con 0x7f2ca4105660 2024-06-02T15:43:02.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.925+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41a88e0 con 0x7f2ca4105660 2024-06-02T15:43:02.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.929+0000 7f2ca1600700 1 -- 172.21.15.90:0/1665563198 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c98029070 con 0x7f2ca4105660 2024-06-02T15:43:02.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.930+0000 7f2cb616cb80 1 Processor -- start 2024-06-02T15:43:02.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 -- start start 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 0x5624ea7893b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea783460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 0x5624ea783ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624ea6e1df0 con 0x5624ea788fd0 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624ea6d1990 con 0x5624ea783a80 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ea6d59d0 con 0x5624ea7898f0 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.931+0000 7f2ca1600700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 0x7f2c8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 0x5624ea7893b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea783460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 0x5624ea7893b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33928/0 (socket says 172.21.15.90:33928) 2024-06-02T15:43:02.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca9800700 1 -- 172.21.15.90:0/3450400114 learned_addr learned my addr 172.21.15.90:0/3450400114 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 0x5624ea783ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca1600700 1 -- 172.21.15.90:0/1665563198 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c980e3140 con 0x7f2ca4105660 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca9800700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 0x7f2c8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3450400114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 msgr2=0x5624ea783460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea783460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3450400114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 msgr2=0x5624ea7893b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 0x5624ea7893b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3450400114 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ea69a3d0 con 0x5624ea783a80 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.932+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea783460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.933+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 0x5624ea783ee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009c60 tx=0x7f2c94009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f3b93ff844675 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.933+0000 7f2ca9800700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 0x7f2c8c0b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009f80 tx=0x7f2c9c004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.933+0000 7f2c92000700 1 -- 172.21.15.90:0/3450400114 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011d10 con 0x5624ea783a80 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.933+0000 7f2c92000700 1 -- 172.21.15.90:0/3450400114 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c94004500 con 0x5624ea783a80 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.933+0000 7f2c92000700 1 -- 172.21.15.90:0/3450400114 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94005280 con 0x5624ea783a80 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3450400114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 msgr2=0x5624ea783ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 0x5624ea783ee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009c60 tx=0x7f2c94009f70 comp rx=0 tx=0).stop 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3450400114 shutdown_connections 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea783a80 0x5624ea783ee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea788fd0 0x5624ea7893b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3450400114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea783460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3450400114 >> 172.21.15.90:0/3450400114 conn(0x5624ea6b1b90 msgr2=0x5624ea77e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3450400114 shutdown_connections 2024-06-02T15:43:02.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.934+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3450400114 wait complete. 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 -- start start 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 Processor -- start 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 0x5624ea78c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea78ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624ea5d7f50 con 0x5624ea783a80 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624ea672710 con 0x5624ea788fd0 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ea790410 con 0x5624ea7898f0 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.935+0000 7f2cb616cb80 10 client.0 _ll_register_callbacks cb 0x5624ea67bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33932/0 (socket says 172.21.15.90:33932) 2024-06-02T15:43:02.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 0x5624ea78c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca8e00700 1 -- 172.21.15.90:0/3283697570 learned_addr learned my addr 172.21.15.90:0/3283697570 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca9800700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea78ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3283697570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 msgr2=0x5624ea78ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea78ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 msgr2=0x5624ea78f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3283697570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ea621880 con 0x5624ea788fd0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 0x5624ea78c970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c94019000 tx=0x7f2c94005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b409ac317debe0ac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.936+0000 7f2c86a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94026020 con 0x5624ea788fd0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c94004500 con 0x5624ea788fd0 2024-06-02T15:43:02.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c94004750 con 0x5624ea788fd0 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c94031050 con 0x5624ea788fd0 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2cb616cb80 10 client.25109 fetch_fsmap learned FSMap version 25 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624ea6a2010 con 0x5624ea788fd0 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.937+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c94004500 con 0x5624ea788fd0 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c94018070 con 0x5624ea788fd0 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 10 client.25109 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 10 client.25109 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 20 client.25109 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea79cd20 0x5624ea79e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 4 client.25109 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:02.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ea6e3d40 con 0x5624ea79cd20 2024-06-02T15:43:02.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.938+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea79cd20 0x5624ea79e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.939+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea79cd20 0x5624ea79e190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f55ee3fcfc1c2bf5 server_cookie=e5245d2658311c29 in_seq=0 out_seq=0 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.939+0000 7f2c86a00700 10 client.25109 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.939+0000 7f2c86a00700 1 -- 172.21.15.90:0/3283697570 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624ea6e3d40 con 0x5624ea79cd20 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.939+0000 7f2c86a00700 10 client.25109 handle_command_reply: tid=0 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.939+0000 7f2cb616cb80 1 client.25109 shutdown 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 msgr2=0x5624ea78c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 0x5624ea78c970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c94019000 tx=0x7f2c94005d60 comp rx=0 tx=0).stop 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 shutdown_connections 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea78ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea78f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea79cd20 0x5624ea79e190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3283697570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea788fd0 0x5624ea78c970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 >> 172.21.15.90:0/3283697570 conn(0x5624ea6b1b90 msgr2=0x5624ea782450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2c86a00700 0 client.25109 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:43:02.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 shutdown_connections 2024-06-02T15:43:02.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3283697570 wait complete. 2024-06-02T15:43:02.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.940+0000 7f2cb616cb80 20 client.25109 trim_cache size 0 max 16384 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.963+0000 7f2cb616cb80 1 Processor -- start 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 -- start start 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea79bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7898f0 0x5624ea787060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79c4d0 con 0x5624ea783a80 2024-06-02T15:43:02.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79cba0 con 0x5624ea7898f0 2024-06-02T15:43:02.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79d270 con 0x5624ea788fd0 2024-06-02T15:43:02.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41486/0 (socket says 172.21.15.90:41486) 2024-06-02T15:43:02.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2ca3e00700 1 -- 172.21.15.90:0/918263822 learned_addr learned my addr 172.21.15.90:0/918263822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2ca9800700 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7898f0 0x5624ea787060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.964+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea79bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 -- 172.21.15.90:0/918263822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7898f0 msgr2=0x5624ea787060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7898f0 0x5624ea787060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 -- 172.21.15.90:0/918263822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 msgr2=0x5624ea79bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea79bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 -- 172.21.15.90:0/918263822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ea78aee0 con 0x5624ea788fd0 2024-06-02T15:43:02.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.965+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004df0 tx=0x7f2c94009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee0daa09ce21477 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.966+0000 7f2c91600700 1 -- 172.21.15.90:0/918263822 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94026040 con 0x5624ea788fd0 2024-06-02T15:43:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.966+0000 7f2c91600700 1 -- 172.21.15.90:0/918263822 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c94018d90 con 0x5624ea788fd0 2024-06-02T15:43:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.966+0000 7f2c91600700 1 -- 172.21.15.90:0/918263822 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003d20 con 0x5624ea788fd0 2024-06-02T15:43:02.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.966+0000 7f2cb616cb80 1 -- 172.21.15.90:0/918263822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 msgr2=0x5624ea786b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.966+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c94004df0 tx=0x7f2c94009f70 comp rx=0 tx=0).stop 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 -- 172.21.15.90:0/918263822 shutdown_connections 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7898f0 0x5624ea787060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea79bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/918263822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea788fd0 0x5624ea786b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 -- 172.21.15.90:0/918263822 >> 172.21.15.90:0/918263822 conn(0x5624ea6b1b90 msgr2=0x5624ea777f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 -- 172.21.15.90:0/918263822 shutdown_connections 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 -- 172.21.15.90:0/918263822 wait complete. 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 -- start start 2024-06-02T15:43:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.967+0000 7f2cb616cb80 1 Processor -- start 2024-06-02T15:43:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea7a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7a25c0 0x5624ea7a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79e010 con 0x5624ea783a80 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79d940 con 0x5624ea7a25c0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ea79e6e0 con 0x5624ea7898f0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2cb616cb80 10 client.0 _ll_register_callbacks cb 0x5624ea67bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33956/0 (socket says 172.21.15.90:33956) 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2ca8e00700 1 -- 172.21.15.90:0/3898835322 learned_addr learned my addr 172.21.15.90:0/3898835322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea7a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.968+0000 7f2ca9800700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7a25c0 0x5624ea7a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 -- 172.21.15.90:0/3898835322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 msgr2=0x5624ea7a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea7a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 -- 172.21.15.90:0/3898835322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7a25c0 msgr2=0x5624ea7a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7a25c0 0x5624ea7a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 -- 172.21.15.90:0/3898835322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ea7a2f80 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b2e0 tx=0x7f2c98005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88981913510b9a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2c86a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980c4060 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c980aa640 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c94018d90 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2c9800f460 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2cb616cb80 10 client.14997 fetch_fsmap learned FSMap version 25 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.969+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624ea811920 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c94018d90 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2c980cb1d0 con 0x5624ea783a80 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 10 client.14997 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:43:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 10 client.14997 resolve_mds: resolved name 'a' to daemon mds.a 2024-06-02T15:43:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 20 client.14997 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea8164e0 0x5624ea8151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 4 client.14997 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:43:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "status"}) -- 0x5624ea6dd520 con 0x5624ea8164e0 2024-06-02T15:43:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.970+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea8164e0 0x5624ea8151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea8164e0 0x5624ea8151f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9a73684bb3fa3aa server_cookie=c1a8c925d9bbac79 in_seq=0 out_seq=0 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2c86a00700 10 client.14997 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2c86a00700 1 -- 172.21.15.90:0/3898835322 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5624ea6dd520 con 0x5624ea8164e0 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2c86a00700 10 client.14997 handle_command_reply: tid=0 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2cb616cb80 1 client.14997 shutdown 2024-06-02T15:43:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 msgr2=0x5624ea7a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800b2e0 tx=0x7f2c98005eb0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 shutdown_connections 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.971+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5624ea8164e0 0x5624ea8151f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ea7a25c0 0x5624ea7a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ea783a80 0x5624ea7a7610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2c86a00700 0 client.14997 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 --2- 172.21.15.90:0/3898835322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ea7898f0 0x5624ea7a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 >> 172.21.15.90:0/3898835322 conn(0x5624ea6b1b90 msgr2=0x5624ea79bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 shutdown_connections 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 1 -- 172.21.15.90:0/3898835322 wait complete. 2024-06-02T15:43:02.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.972+0000 7f2cb616cb80 20 client.14997 trim_cache size 0 max 16384 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24524, 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 211.075217235, 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:43:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 308.823988671 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 msgr2=0x7f2c8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 0x7f2c8c0b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009f80 tx=0x7f2c9c004b40 comp rx=0 tx=0).stop 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 msgr2=0x7f2ca419bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c980094a0 tx=0x7f2c98004c70 comp rx=0 tx=0).stop 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 shutdown_connections 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b2900 0x7f2c8c0b4dc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca410b660 0x7f2ca41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4105660 0x7f2ca419bd60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 --2- 172.21.15.90:0/1665563198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4106020 0x7f2ca419c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 >> 172.21.15.90:0/1665563198 conn(0x7f2ca41010f0 msgr2=0x7f2ca41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 shutdown_connections 2024-06-02T15:43:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:02.973+0000 7f2caa200700 1 -- 172.21.15.90:0/1665563198 wait complete. 2024-06-02T15:43:03.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:03 smithi090 ceph-mon[31595]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:03 smithi120 ceph-mon[36451]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:03 smithi135 ceph-mon[36839]: pgmap v472: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:04.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T15:43:04.490 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:05.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:05 smithi090 ceph-mon[31595]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:05 smithi120 ceph-mon[36451]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:05 smithi135 ceph-mon[36839]: pgmap v473: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:06.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.826+0000 7fae97a00700 1 -- 172.21.15.90:0/2551512177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901088f0 msgr2=0x7fae90110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.826+0000 7fae97a00700 1 --2- 172.21.15.90:0/2551512177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901088f0 0x7fae90110470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fae8800b0d0 tx=0x7fae8800b3e0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 -- 172.21.15.90:0/2551512177 shutdown_connections 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 --2- 172.21.15.90:0/2551512177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901088f0 0x7fae90110470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 --2- 172.21.15.90:0/2551512177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901037f0 0x7fae90103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 --2- 172.21.15.90:0/2551512177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae901081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 -- 172.21.15.90:0/2551512177 >> 172.21.15.90:0/2551512177 conn(0x7fae9007a6a0 msgr2=0x7fae9007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 -- 172.21.15.90:0/2551512177 shutdown_connections 2024-06-02T15:43:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.827+0000 7fae97a00700 1 -- 172.21.15.90:0/2551512177 wait complete. 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 Processor -- start 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 -- start start 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901037f0 0x7fae9019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901088f0 0x7fae901a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae90113730 con 0x7fae901041a0 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae901135b0 con 0x7fae901037f0 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.828+0000 7fae97a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae901138b0 con 0x7fae901088f0 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33976/0 (socket says 172.21.15.90:33976) 2024-06-02T15:43:06.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 -- 172.21.15.90:0/542359485 learned_addr learned my addr 172.21.15.90:0/542359485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae95a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901088f0 0x7fae901a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae95000700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901037f0 0x7fae9019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 -- 172.21.15.90:0/542359485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901088f0 msgr2=0x7fae901a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901088f0 0x7fae901a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 -- 172.21.15.90:0/542359485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901037f0 msgr2=0x7fae9019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901037f0 0x7fae9019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.829+0000 7fae8fe00700 1 -- 172.21.15.90:0/542359485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae901a0dc0 con 0x7fae901041a0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae8fe00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00eb30 tx=0x7fae7c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae8d600700 1 -- 172.21.15.90:0/542359485 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00cbd0 con 0x7fae901041a0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae901a10b0 con 0x7fae901041a0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae8d600700 1 -- 172.21.15.90:0/542359485 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c0044d0 con 0x7fae901041a0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae8d600700 1 -- 172.21.15.90:0/542359485 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c003e30 con 0x7fae901041a0 2024-06-02T15:43:06.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.830+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae901a8930 con 0x7fae901041a0 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.833+0000 7fae8d600700 1 -- 172.21.15.90:0/542359485 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae7c004000 con 0x7fae901041a0 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.834+0000 7faea2308b80 1 Processor -- start 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 -- start start 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7fae8d600700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 0x7fae780b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b966210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b966830 0x55a51b966c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a51b8c4df0 con 0x55a51b96be60 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a51b8b4990 con 0x55a51b966830 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7faea2308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a51b8b89d0 con 0x55a51b96c780 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7fae8d600700 1 -- 172.21.15.90:0/542359485 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fae7c0e2910 con 0x7fae901041a0 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.835+0000 7fae95a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 0x7fae780b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33990/0 (socket says 172.21.15.90:33990) 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95a00700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b966830 0x55a51b966c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95000700 1 -- 172.21.15.90:0/342611588 learned_addr learned my addr 172.21.15.90:0/342611588 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b966210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 -- 172.21.15.90:0/342611588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b966830 msgr2=0x55a51b966c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b966830 0x55a51b966c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 -- 172.21.15.90:0/342611588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 msgr2=0x55a51b96c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 -- 172.21.15.90:0/342611588 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51b87d3d0 con 0x55a51b96c780 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95000700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae95a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 0x7fae780b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae88009fd0 tx=0x7fae88009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.836+0000 7fae8fe00700 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b966210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0e0000 tx=0x7fae7c0a59e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75dcc12688a849c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.837+0000 7fae82000700 1 -- 172.21.15.90:0/342611588 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0c3070 con 0x55a51b96c780 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.837+0000 7fae82000700 1 -- 172.21.15.90:0/342611588 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c0aae50 con 0x55a51b96c780 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.837+0000 7fae82000700 1 -- 172.21.15.90:0/342611588 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0a9fe0 con 0x55a51b96c780 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- 172.21.15.90:0/342611588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 msgr2=0x55a51b966210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b966210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0e0000 tx=0x7fae7c0a59e0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- 172.21.15.90:0/342611588 shutdown_connections 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b966830 0x55a51b966c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b96c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 --2- 172.21.15.90:0/342611588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b966210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- 172.21.15.90:0/342611588 >> 172.21.15.90:0/342611588 conn(0x55a51b894b90 msgr2=0x55a51b960fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- 172.21.15.90:0/342611588 shutdown_connections 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- 172.21.15.90:0/342611588 wait complete. 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.838+0000 7faea2308b80 1 -- start start 2024-06-02T15:43:06.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 Processor -- start 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 0x55a51b96f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b96fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a51b7baf50 con 0x55a51b966830 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a51b855710 con 0x55a51b96be60 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a51b9731f0 con 0x55a51b96c780 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.839+0000 7faea2308b80 10 client.0 _ll_register_callbacks cb 0x55a51b85ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34002/0 (socket says 172.21.15.90:34002) 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 -- 172.21.15.90:0/3302924518 learned_addr learned my addr 172.21.15.90:0/3302924518 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95a00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b96fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae8fe00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 0x55a51b96f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 -- 172.21.15.90:0/3302924518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 msgr2=0x55a51b96fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b96fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 -- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 msgr2=0x55a51b96f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 0x55a51b96f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 -- 172.21.15.90:0/3302924518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51b804880 con 0x55a51b966830 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95a00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b96fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae8fe00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 0x55a51b96f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae95000700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae84012840 tx=0x7fae84012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0735a4f48c537e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.840+0000 7fae76a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400f540 con 0x55a51b966830 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fae8400f6e0 con 0x55a51b966830 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae7c0aae50 con 0x55a51b966830 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae8400f8e0 con 0x55a51b966830 2024-06-02T15:43:06.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7faea2308b80 10 client.15009 fetch_fsmap learned FSMap version 25 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a51b885010 con 0x55a51b966830 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.841+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c0aae50 con 0x55a51b966830 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae84010a00 con 0x55a51b966830 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 10 client.15009 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 10 client.15009 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 20 client.15009 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b97fe90 0x55a51b981300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 4 client.15009 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a51b8c6d40 con 0x55a51b97fe90 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.842+0000 7fae8fe00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b97fe90 0x55a51b981300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.843+0000 7fae8fe00700 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b97fe90 0x55a51b981300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4744f962658ecdad server_cookie=9b737d5c088abb97 in_seq=0 out_seq=0 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.843+0000 7fae76a00700 10 client.15009 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.843+0000 7fae76a00700 1 -- 172.21.15.90:0/3302924518 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a51b8c6d40 con 0x55a51b97fe90 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.843+0000 7fae76a00700 10 client.15009 handle_command_reply: tid=0 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.843+0000 7faea2308b80 1 client.15009 shutdown 2024-06-02T15:43:06.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 msgr2=0x55a51b972580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae84012840 tx=0x7fae84012b50 comp rx=0 tx=0).stop 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 shutdown_connections 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b97fe90 0x55a51b981300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96c780 0x55a51b96fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b972580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 --2- 172.21.15.90:0/3302924518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96be60 0x55a51b96f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7fae76a00700 0 client.15009 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 >> 172.21.15.90:0/3302924518 conn(0x55a51b894b90 msgr2=0x55a51b965200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 shutdown_connections 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 1 -- 172.21.15.90:0/3302924518 wait complete. 2024-06-02T15:43:06.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.844+0000 7faea2308b80 20 client.15009 trim_cache size 0 max 16384 2024-06-02T15:43:06.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 Processor -- start 2024-06-02T15:43:06.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 -- start start 2024-06-02T15:43:06.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96be60 0x55a51b971c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b969970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a51b97f640 con 0x55a51b966830 2024-06-02T15:43:06.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a51b97fd10 con 0x55a51b96c780 2024-06-02T15:43:06.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.867+0000 7faea2308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a51b9803e0 con 0x55a51b96be60 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34014/0 (socket says 172.21.15.90:34014) 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 -- 172.21.15.90:0/3359571202 learned_addr learned my addr 172.21.15.90:0/3359571202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae8fe00700 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96be60 0x55a51b971c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95a00700 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b969970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 -- 172.21.15.90:0/3359571202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96be60 msgr2=0x55a51b971c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96be60 0x55a51b971c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 -- 172.21.15.90:0/3359571202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 msgr2=0x55a51b969970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b969970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 -- 172.21.15.90:0/3359571202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51b96bab0 con 0x55a51b966830 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.868+0000 7fae95000700 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae8400f7d0 tx=0x7fae8400f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564bef9fcd93835e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7fae81600700 1 -- 172.21.15.90:0/3359571202 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8402f070 con 0x55a51b966830 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7fae81600700 1 -- 172.21.15.90:0/3359571202 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae840050e0 con 0x55a51b966830 2024-06-02T15:43:06.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7fae81600700 1 -- 172.21.15.90:0/3359571202 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8401a5b0 con 0x55a51b966830 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 -- 172.21.15.90:0/3359571202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 msgr2=0x55a51b966210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae8400f7d0 tx=0x7fae8400f800 comp rx=0 tx=0).stop 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 -- 172.21.15.90:0/3359571202 shutdown_connections 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b969970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b966830 0x55a51b966210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.869+0000 7faea2308b80 1 --2- 172.21.15.90:0/3359571202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b96be60 0x55a51b971c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.870+0000 7faea2308b80 1 -- 172.21.15.90:0/3359571202 >> 172.21.15.90:0/3359571202 conn(0x55a51b894b90 msgr2=0x55a51b96e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.870+0000 7faea2308b80 1 -- 172.21.15.90:0/3359571202 shutdown_connections 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.870+0000 7faea2308b80 1 -- 172.21.15.90:0/3359571202 wait complete. 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.870+0000 7faea2308b80 1 -- start start 2024-06-02T15:43:06.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.870+0000 7faea2308b80 1 Processor -- start 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b970c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b9711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a51b981180 con 0x55a51b96be60 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a51b980ab0 con 0x55a51b96c780 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a51b981850 con 0x55a51b966830 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7faea2308b80 10 client.0 _ll_register_callbacks cb 0x55a51b85ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b970c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b9711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41542/0 (socket says 172.21.15.90:41542) 2024-06-02T15:43:06.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 -- 172.21.15.90:0/2604267420 learned_addr learned my addr 172.21.15.90:0/2604267420 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 -- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 msgr2=0x55a51b9711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b9711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 -- 172.21.15.90:0/2604267420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 msgr2=0x55a51b970c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.871+0000 7fae95000700 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b970c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae95000700 1 -- 172.21.15.90:0/2604267420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51b971a10 con 0x55a51b966830 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae95000700 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae84012b90 tx=0x7fae840054c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe146603621d1e45 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:06.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae76a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8402f070 con 0x55a51b966830 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae840050e0 con 0x55a51b966830 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fae8401a370 con 0x55a51b966830 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae8401a570 con 0x55a51b966830 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7faea2308b80 10 client.25102 fetch_fsmap learned FSMap version 25 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.872+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a51b9eb6e0 con 0x55a51b966830 2024-06-02T15:43:06.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae840050e0 con 0x55a51b966830 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fae84003a30 con 0x55a51b966830 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 10 client.25102 fetch_fsmap finished waiting for FSMap version 25 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 10 client.25102 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 20 client.25102 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b9f00d0 0x55a51b9eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 4 client.25102 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.873+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "status"}) -- 0x55a51b8c0520 con 0x55a51b9f00d0 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7fae8fe00700 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b9f00d0 0x55a51b9eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7fae8fe00700 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b9f00d0 0x55a51b9eede0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea5f79789b03148 server_cookie=81db9effebdb0519 in_seq=0 out_seq=0 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7fae76a00700 10 client.25102 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7fae76a00700 1 -- 172.21.15.90:0/2604267420 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a51b8c0520 con 0x55a51b9f00d0 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7fae76a00700 10 client.25102 handle_command_reply: tid=0 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7faea2308b80 1 client.25102 shutdown 2024-06-02T15:43:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 msgr2=0x55a51b98af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.874+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae84012b90 tx=0x7fae840054c0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 shutdown_connections 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a51b9f00d0 0x55a51b9eede0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a51b96c780 0x55a51b9711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a51b966830 0x55a51b98af40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 --2- 172.21.15.90:0/2604267420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a51b96be60 0x55a51b970c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7fae76a00700 0 client.25102 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 >> 172.21.15.90:0/2604267420 conn(0x55a51b894b90 msgr2=0x55a51b96e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 shutdown_connections 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 1 -- 172.21.15.90:0/2604267420 wait complete. 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7faea2308b80 20 client.25102 trim_cache size 0 max 16384 2024-06-02T15:43:06.875 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "935be7dc-20f4-11ef-bc9b-c7b262605968", 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24536, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 287.39512623299998, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 309.845641796 2024-06-02T15:43:06.876 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 msgr2=0x7fae780b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 0x7fae780b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae88009fd0 tx=0x7fae88009440 comp rx=0 tx=0).stop 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.875+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 msgr2=0x7fae9019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00eb30 tx=0x7fae7c00ee40 comp rx=0 tx=0).stop 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 shutdown_connections 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae780b2900 0x7fae780b4dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae901088f0 0x7fae901a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae901037f0 0x7fae9019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 --2- 172.21.15.90:0/542359485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae901041a0 0x7fae9019c340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 >> 172.21.15.90:0/542359485 conn(0x7fae9007a6a0 msgr2=0x7fae901015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 shutdown_connections 2024-06-02T15:43:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:06.876+0000 7fae97a00700 1 -- 172.21.15.90:0/542359485 wait complete. 2024-06-02T15:43:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:07 smithi120 ceph-mon[36451]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:07 smithi135 ceph-mon[36839]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:07 smithi090 ceph-mon[31595]: pgmap v474: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:07.830 INFO:teuthology.run_tasks:Running task sequential... 2024-06-02T15:43:07.838 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-02T15:43:07.838 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-02T15:43:07.839 INFO:teuthology.task.print:Enabling mgr modules 2024-06-02T15:43:07.839 INFO:teuthology.run_tasks:Running task kclient... 2024-06-02T15:43:07.876 INFO:tasks.kclient:Mounting kernel clients... 2024-06-02T15:43:07.876 INFO:tasks.kclient:config is {'client.0': {}} 2024-06-02T15:43:07.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:07.876 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-06-02T15:43:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-02T15:43:07.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:07.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-02T15:43:08.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:08.033 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-06-02T15:43:08.148 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-06-02T15:43:08.149 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-02T15:43:08.149 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:08.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-02T15:43:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:09.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:09 smithi090 ceph-mon[31595]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:09 smithi120 ceph-mon[36451]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:09 smithi135 ceph-mon[36839]: pgmap v475: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:10.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 -- 172.21.15.90:0/1813079181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 msgr2=0x7fa370116870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:10.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 --2- 172.21.15.90:0/1813079181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa370116870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa36800b0a0 tx=0x7fa36800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 -- 172.21.15.90:0/1813079181 shutdown_connections 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 --2- 172.21.15.90:0/1813079181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa370116870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 --2- 172.21.15.90:0/1813079181 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3700758e0 0x7fa370075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 --2- 172.21.15.90:0/1813079181 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa370076200 0x7fa370113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.852+0000 7fa376200700 1 -- 172.21.15.90:0/1813079181 >> 172.21.15.90:0/1813079181 conn(0x7fa3700fefc0 msgr2=0x7fa3701013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.853+0000 7fa376200700 1 -- 172.21.15.90:0/1813079181 shutdown_connections 2024-06-02T15:43:10.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.853+0000 7fa376200700 1 -- 172.21.15.90:0/1813079181 wait complete. 2024-06-02T15:43:10.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 Processor -- start 2024-06-02T15:43:10.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 -- start start 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa370076200 0x7fa3701a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa3701a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa370119c70 con 0x7fa370076200 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa370119af0 con 0x7fa3700758e0 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.854+0000 7fa376200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa370119df0 con 0x7fa370114480 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa367e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa370076200 0x7fa3701a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35324/0 (socket says 172.21.15.90:35324) 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 -- 172.21.15.90:0/1873865507 learned_addr learned my addr 172.21.15.90:0/1873865507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa375800700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa3701a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:10.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 -- 172.21.15.90:0/1873865507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 msgr2=0x7fa3701a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa3701a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 -- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa370076200 msgr2=0x7fa3701a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa370076200 0x7fa3701a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.855+0000 7fa374e00700 1 -- 172.21.15.90:0/1873865507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3701a5130 con 0x7fa3700758e0 2024-06-02T15:43:10.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa374e00700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9a0 tx=0x7fa36000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ca10 con 0x7fa3700758e0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3701a5420 con 0x7fa3700758e0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa36000cbb0 con 0x7fa3700758e0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3701acca0 con 0x7fa3700758e0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.856+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360018420 con 0x7fa3700758e0 2024-06-02T15:43:10.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.857+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37006a230 con 0x7fa3700758e0 2024-06-02T15:43:10.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.859+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa360010430 con 0x7fa3700758e0 2024-06-02T15:43:10.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.861+0000 7fa36e000700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 0x7fa3540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:10.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.861+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa3600e29b0 con 0x7fa3700758e0 2024-06-02T15:43:10.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.861+0000 7fa367e00700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 0x7fa3540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:10.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.862+0000 7fa367e00700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 0x7fa3540b4dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa358009680 tx=0x7fa358009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:10.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:10.863+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3600aa2c0 con 0x7fa3700758e0 2024-06-02T15:43:11.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.148+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa37002e560 con 0x7fa3700758e0 2024-06-02T15:43:11.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.149+0000 7fa36e000700 1 -- 172.21.15.90:0/1873865507 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7fa36000cd60 con 0x7fa3700758e0 2024-06-02T15:43:11.146 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-02T15:43:11.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 msgr2=0x7fa3540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:11.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 0x7fa3540b4dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa358009680 tx=0x7fa358009260 comp rx=0 tx=0).stop 2024-06-02T15:43:11.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 msgr2=0x7fa3701a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9a0 tx=0x7fa36000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 shutdown_connections 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa3540b2900 0x7fa3540b4dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa370114480 0x7fa3701a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa3700758e0 0x7fa3701a0120 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 --2- 172.21.15.90:0/1873865507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa370076200 0x7fa3701a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 >> 172.21.15.90:0/1873865507 conn(0x7fa3700fefc0 msgr2=0x7fa370100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 shutdown_connections 2024-06-02T15:43:11.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:11.152+0000 7fa376200700 1 -- 172.21.15.90:0/1873865507 wait complete. 2024-06-02T15:43:11.519 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:11 smithi090 ceph-mon[31595]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:11.519 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1873865507' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:43:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:11 smithi120 ceph-mon[36451]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1873865507' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:43:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:11 smithi135 ceph-mon[36839]: pgmap v476: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1873865507' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T15:43:12.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-02T15:43:12.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-02T15:43:12.049 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-02T15:43:12.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:12.049 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-06-02T15:43:12.106 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:43:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41674sec preferred_lft 41674sec 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-06-02T15:43:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-06-02T15:43:12.109 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-02T15:43:12.109 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:43:12.109 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:43:12.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-06-02T15:43:12.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-06-02T15:43:12.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-06-02T15:43:12.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-02T15:43:12.110 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:43:12.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:12.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:12.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:12.353 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-02T15:43:12.422 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-06-02T15:43:12.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:12.425 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-06-02T15:43:12.479 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-06-02T15:43:12.479 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-02T15:43:12.479 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-02T15:43:12.479 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-02T15:43:12.479 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-02T15:43:12.479 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:43:12.479 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:43:12.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-02T15:43:12.480 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-02T15:43:12.480 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-02T15:43:12.480 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:43:12.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:12.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:12.680 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-06-02T15:43:12.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:12.726 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-06-02T15:43:12.822 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:43:12.822 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:43:12.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T15:43:12.823 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-02T15:43:12.823 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:43:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:13.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:13.013 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-02T15:43:13.013 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:43:13.013 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:43:13.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-02T15:43:13.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-02T15:43:13.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-02T15:43:13.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-02T15:43:13.014 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-02T15:43:13.014 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:43:13.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:13.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:13.288 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T15:43:13.288 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-06-02T15:43:13.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-06-02T15:43:13.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-06-02T15:43:13.288 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-06-02T15:43:13.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:13.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:13 smithi090 ceph-mon[31595]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:13.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:13.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T15:43:13.488 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:13.531 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-02T15:43:13.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T15:43:13.531 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:13 smithi120 ceph-mon[36451]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:13 smithi135 ceph-mon[36839]: pgmap v477: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-06-02T15:43:13.625 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-06-02T15:43:13.625 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-06-02T15:43:13.626 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6c423124-5043-4aa3-b141-d9bef9e296fb 2024-06-02T15:43:13.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:13.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6c423124-5043-4aa3-b141-d9bef9e296fb /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-02T15:43:13.751 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-02T15:43:13.751 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-02T15:43:13.751 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-06-02T15:43:14.550 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-06-02T15:43:14.551 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-06-02T15:43:14.551 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-06-02T15:43:14.551 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.120:3300,172.21.15.135:3300" 2024-06-02T15:43:14.551 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.90:3300,172.21.15.120:3300,172.21.15.135:3300:/volumes/qa/sv_1/6c423124-5043-4aa3-b141-d9bef9e296fb 2024-06-02T15:43:14.551 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=935be7dc-20f4-11ef-bc9b-c7b262605968" will pass to kernel 2024-06-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-06-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-06-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-06-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-06-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-06-02T15:43:14.558 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-06-02T15:43:14.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:43:14.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:14.634 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-02T15:43:14.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-02T15:43:14.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:14.755 INFO:teuthology.orchestra.run.smithi090.stdout:client25111 2024-06-02T15:43:14.755 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:43:14.755 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T15:43:14.807 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-02T15:43:14.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-02T15:43:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:14.926 INFO:teuthology.orchestra.run.smithi090.stdout:client25111 2024-06-02T15:43:14.927 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-06-02T15:43:15.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-02T15:43:15.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:15.042 INFO:teuthology.orchestra.run.smithi090.stdout:client25111 2024-06-02T15:43:15.042 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968.client25111/status') 2024-06-02T15:43:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T15:43:15.174 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.25111 (3)172.21.15.90:0/2098902359 2024-06-02T15:43:15.174 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-06-02T15:43:15.174 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-06-02T15:43:15.174 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-06-02T15:43:15.174 INFO:teuthology.orchestra.run.smithi090.stderr:71 bytes copied, 8.2125e-05 s, 865 kB/s 2024-06-02T15:43:15.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T15:43:15.187 INFO:teuthology.run_tasks:Running task exec... 2024-06-02T15:43:15.198 INFO:teuthology.task.exec:Executing custom commands... 2024-06-02T15:43:15.198 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:43:15.198 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-02T15:43:15.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 -- 172.21.15.90:0/3707427528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc104ce0 msgr2=0x7fe6fc105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 --2- 172.21.15.90:0/3707427528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc105100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009860 tx=0x7fe6ec02a340 comp rx=0 tx=0).stop 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 -- 172.21.15.90:0/3707427528 shutdown_connections 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 --2- 172.21.15.90:0/3707427528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc106890 0x7fe6fc10d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 --2- 172.21.15.90:0/3707427528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 --2- 172.21.15.90:0/3707427528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc105100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 -- 172.21.15.90:0/3707427528 >> 172.21.15.90:0/3707427528 conn(0x7fe6fc100270 msgr2=0x7fe6fc102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 -- 172.21.15.90:0/3707427528 shutdown_connections 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.400+0000 7fe703600640 1 -- 172.21.15.90:0/3707427528 wait complete. 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.401+0000 7fe703600640 1 Processor -- start 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.401+0000 7fe703600640 1 -- start start 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 0x7fe6fc1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:15.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc110810 con 0x7fe6fc105ed0 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc110690 con 0x7fe6fc106890 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe703600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc110990 con 0x7fe6fc104ce0 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe6f3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe701600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 0x7fe6fc1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe6f3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47728/0 (socket says 172.21.15.90:47728) 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe6f3e00640 1 -- 172.21.15.90:0/3264042682 learned_addr learned my addr 172.21.15.90:0/3264042682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe700c00640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe701600640 1 -- 172.21.15.90:0/3264042682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 msgr2=0x7fe6fc19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe701600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.402+0000 7fe701600640 1 -- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 msgr2=0x7fe6fc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:15.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe701600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe701600640 1 -- 172.21.15.90:0/3264042682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1a2a70 con 0x7fe6fc106890 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe700c00640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe701600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 0x7fe6fc1a2400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009fd0 tx=0x7fe6f400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400ce20 con 0x7fe6fc106890 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc1a2d60 con 0x7fe6fc106890 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f4005020 con 0x7fe6fc106890 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4010690 con 0x7fe6fc106890 2024-06-02T15:43:15.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.403+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc1a3250 con 0x7fe6fc106890 2024-06-02T15:43:15.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.404+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc10c000 con 0x7fe6fc106890 2024-06-02T15:43:15.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.407+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe6f40054d0 con 0x7fe6fc106890 2024-06-02T15:43:15.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.408+0000 7fe6fa000640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 0x7fe6e00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:15.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.408+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe6f40e2fc0 con 0x7fe6fc106890 2024-06-02T15:43:15.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.408+0000 7fe700c00640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 0x7fe6e00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:15.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.409+0000 7fe700c00640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 0x7fe6e00b4d60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009830 tx=0x7fe6ec030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:15.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.410+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6f40aa8d0 con 0x7fe6fc106890 2024-06-02T15:43:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:15 smithi120 ceph-mon[36451]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:15 smithi135 ceph-mon[36839]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:15 smithi090 ceph-mon[31595]: pgmap v478: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-06-02T15:43:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:15.664+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fe6fc02dba0 con 0x7fe6fc106890 2024-06-02T15:43:16.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.244+0000 7fe6fa000640 1 -- 172.21.15.90:0/3264042682 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7fe6f40aa1d0 con 0x7fe6fc106890 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.248+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 msgr2=0x7fe6e00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.248+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 0x7fe6e00b4d60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009830 tx=0x7fe6ec030450 comp rx=0 tx=0).stop 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.248+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 msgr2=0x7fe6fc1a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.248+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 0x7fe6fc1a2400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009fd0 tx=0x7fe6f400ef10 comp rx=0 tx=0).stop 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 shutdown_connections 2024-06-02T15:43:16.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe6e00b28a0 0x7fe6e00b4d60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe6fc106890 0x7fe6fc1a2400 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6fc105ed0 0x7fe6fc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 --2- 172.21.15.90:0/3264042682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6fc104ce0 0x7fe6fc19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 >> 172.21.15.90:0/3264042682 conn(0x7fe6fc100270 msgr2=0x7fe6fc1091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 shutdown_connections 2024-06-02T15:43:16.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.249+0000 7fe703600640 1 -- 172.21.15.90:0/3264042682 wait complete. 2024-06-02T15:43:16.271 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-02T15:43:16.282 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-02T15:43:16.304 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.523+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1124575265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8105cc0 msgr2=0x7fb4a8106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.523+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1124575265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8105cc0 0x7fb4a8106140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb490009900 tx=0x7fb49002a400 comp rx=0 tx=0).stop 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1124575265 shutdown_connections 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1124575265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8106680 0x7fb4a810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1124575265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8105cc0 0x7fb4a8106140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1124575265 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8104ad0 0x7fb4a8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1124575265 >> 172.21.15.90:0/1124575265 conn(0x7fb4a8100060 msgr2=0x7fb4a8102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1124575265 shutdown_connections 2024-06-02T15:43:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.524+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1124575265 wait complete. 2024-06-02T15:43:16.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 Processor -- start 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 -- start start 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 0x7fb4a819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8106680 0x7fb4a81a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8110630 con 0x7fb4a8105cc0 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81104b0 con 0x7fb4a8106680 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81107b0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb49e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb49e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47752/0 (socket says 172.21.15.90:47752) 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb49e000640 1 -- 172.21.15.90:0/1952616564 learned_addr learned my addr 172.21.15.90:0/1952616564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a6a00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 0x7fb4a819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.525+0000 7fb4a7400640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8106680 0x7fb4a81a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 -- 172.21.15.90:0/1952616564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8106680 msgr2=0x7fb4a81a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8106680 0x7fb4a81a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 -- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 msgr2=0x7fb4a819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 -- 172.21.15.90:0/1952616564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a2940 con 0x7fb4a8104ad0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb49e000640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.526+0000 7fb4a6a00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 0x7fb4a819ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e9c0 tx=0x7fb49800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.527+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800ccc0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.527+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a2c30 con 0x7fb4a8104ad0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.527+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4980035f0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.527+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800ce60 con 0x7fb4a8104ad0 2024-06-02T15:43:16.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.527+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a31d0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.528+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a810be70 con 0x7fb4a8104ad0 2024-06-02T15:43:16.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.530+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb498004000 con 0x7fb4a8104ad0 2024-06-02T15:43:16.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.532+0000 7fb49fe00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 0x7fb48c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.532+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb4980e29d0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.532+0000 7fb49e000640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 0x7fb48c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.533+0000 7fb49e000640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 0x7fb48c0b4d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb4900096c0 tx=0x7fb490005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:16.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.534+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4980aa2e0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:16 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3264042682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:16 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:16 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3264042682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:16 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3264042682' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:16 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T15:43:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.742+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4a802dba0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.744+0000 7fb49fe00640 1 -- 172.21.15.90:0/1952616564 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7fb4980a9be0 con 0x7fb4a8104ad0 2024-06-02T15:43:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:16.742 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 msgr2=0x7fb48c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 0x7fb48c0b4d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb4900096c0 tx=0x7fb490005a20 comp rx=0 tx=0).stop 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 msgr2=0x7fb4a819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 0x7fb4a819ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb49800e9c0 tx=0x7fb49800edc0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 shutdown_connections 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb48c0b2870 0x7fb48c0b4d30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4a8106680 0x7fb4a81a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4a8105cc0 0x7fb4a819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 --2- 172.21.15.90:0/1952616564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4a8104ad0 0x7fb4a819ad30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 >> 172.21.15.90:0/1952616564 conn(0x7fb4a8100060 msgr2=0x7fb4a8109040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:16.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 shutdown_connections 2024-06-02T15:43:16.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.747+0000 7fb4a7e00640 1 -- 172.21.15.90:0/1952616564 wait complete. 2024-06-02T15:43:16.764 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T15:32:41.694560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-02T15:38:10.691399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-02T15:38:15.361392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-02T15:38:27.767622+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-02T15:43:16.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.972+0000 7f1357000640 1 -- 172.21.15.90:0/4245788714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 msgr2=0x7f13501114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.972+0000 7f1357000640 1 --2- 172.21.15.90:0/4245788714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501114f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1348009ca0 tx=0x7f134802a8c0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.972+0000 7f1357000640 1 -- 172.21.15.90:0/4245788714 shutdown_connections 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.972+0000 7f1357000640 1 --2- 172.21.15.90:0/4245788714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501114f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.973+0000 7f1357000640 1 --2- 172.21.15.90:0/4245788714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 0x7f135010a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.973+0000 7f1357000640 1 --2- 172.21.15.90:0/4245788714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f13501094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.973+0000 7f1357000640 1 -- 172.21.15.90:0/4245788714 >> 172.21.15.90:0/4245788714 conn(0x7f1350078050 msgr2=0x7f135007a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:16.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.973+0000 7f1357000640 1 -- 172.21.15.90:0/4245788714 shutdown_connections 2024-06-02T15:43:16.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.973+0000 7f1357000640 1 -- 172.21.15.90:0/4245788714 wait complete. 2024-06-02T15:43:16.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 Processor -- start 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 -- start start 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f135019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 0x7f135019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1350114c40 con 0x7f135010ac60 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1350114ac0 con 0x7f13501090b0 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1357000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1350114dc0 con 0x7f135010a2a0 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1355000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f134fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f135019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1355000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501a6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47770/0 (socket says 172.21.15.90:47770) 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1355000640 1 -- 172.21.15.90:0/2034074655 learned_addr learned my addr 172.21.15.90:0/2034074655 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.974+0000 7f1347400640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 0x7f135019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 -- 172.21.15.90:0/2034074655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 msgr2=0x7f135019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 0x7f135019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 -- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 msgr2=0x7f13501a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 -- 172.21.15.90:0/2034074655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13501a6eb0 con 0x7f13501090b0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134fe00640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f135019f290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f134000e9c0 tx=0x7f134000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134000cd80 con 0x7f13501090b0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13501a71a0 con 0x7f13501090b0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13400108e0 con 0x7f13501090b0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.975+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400187f0 con 0x7f13501090b0 2024-06-02T15:43:16.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.976+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501a76f0 con 0x7f13501090b0 2024-06-02T15:43:16.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.977+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13501103d0 con 0x7f13501090b0 2024-06-02T15:43:16.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.979+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f134001f020 con 0x7f13501090b0 2024-06-02T15:43:16.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.980+0000 7f134d600640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 0x7f13340b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:16.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.981+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f13400e2b70 con 0x7f13501090b0 2024-06-02T15:43:16.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.981+0000 7f1347400640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 0x7f13340b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:16.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.981+0000 7f1347400640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 0x7f13340b4cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13501a07a0 tx=0x7f1338005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:16.983+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f13400aa480 con 0x7f13501090b0 2024-06-02T15:43:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.192+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1350063650 con 0x7f13501090b0 2024-06-02T15:43:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.193+0000 7f134d600640 1 -- 172.21.15.90:0/2034074655 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f13400a9d80 con 0x7f13501090b0 2024-06-02T15:43:17.191 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-06-02T15:43:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 msgr2=0x7f13340b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 0x7f13340b4cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13501a07a0 tx=0x7f1338005f90 comp rx=0 tx=0).stop 2024-06-02T15:43:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 msgr2=0x7f135019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f135019f290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f134000e9c0 tx=0x7f134000ee80 comp rx=0 tx=0).stop 2024-06-02T15:43:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 shutdown_connections 2024-06-02T15:43:17.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f13340b27f0 0x7f13340b4cb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f135010ac60 0x7f13501a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f135010a2a0 0x7f135019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 --2- 172.21.15.90:0/2034074655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f13501090b0 0x7f135019f290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 >> 172.21.15.90:0/2034074655 conn(0x7f1350078050 msgr2=0x7f135010d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:17.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 shutdown_connections 2024-06-02T15:43:17.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.196+0000 7f1357000640 1 -- 172.21.15.90:0/2034074655 wait complete. 2024-06-02T15:43:17.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-02T15:43:17.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 -- 172.21.15.90:0/924560002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff01045c0 msgr2=0x7faff010ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 --2- 172.21.15.90:0/924560002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff01045c0 0x7faff010ae50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009cd0 tx=0x7fafe802a8f0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 -- 172.21.15.90:0/924560002 shutdown_connections 2024-06-02T15:43:17.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 --2- 172.21.15.90:0/924560002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff01045c0 0x7faff010ae50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 --2- 172.21.15.90:0/924560002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff0103c00 0x7faff0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 --2- 172.21.15.90:0/924560002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0100c30 0x7faff0101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 -- 172.21.15.90:0/924560002 >> 172.21.15.90:0/924560002 conn(0x7faff00fc580 msgr2=0x7faff00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 -- 172.21.15.90:0/924560002 shutdown_connections 2024-06-02T15:43:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.413+0000 7fafef200640 1 -- 172.21.15.90:0/924560002 wait complete. 2024-06-02T15:43:17.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 Processor -- start 2024-06-02T15:43:17.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 -- start start 2024-06-02T15:43:17.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0103c00 0x7faff019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 0x7faff01a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faff010e510 con 0x7faff0100c30 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faff010e390 con 0x7faff0103c00 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.414+0000 7fafef200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faff010e690 con 0x7faff01045c0 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafee800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 0x7faff01a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47792/0 (socket says 172.21.15.90:47792) 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafee800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 0x7faff01a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55404/0 (socket says 172.21.15.90:55404) 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 -- 172.21.15.90:0/1430693631 learned_addr learned my addr 172.21.15.90:0/1430693631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafed400640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0103c00 0x7faff019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 -- 172.21.15.90:0/1430693631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 msgr2=0x7faff01a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 0x7faff01a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 -- 172.21.15.90:0/1430693631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0103c00 msgr2=0x7faff019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0103c00 0x7faff019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 -- 172.21.15.90:0/1430693631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01a2a10 con 0x7faff0100c30 2024-06-02T15:43:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.415+0000 7fafede00640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e9c0 tx=0x7fafe000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.416+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe000cde0 con 0x7faff0100c30 2024-06-02T15:43:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.416+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe00108e0 con 0x7faff0100c30 2024-06-02T15:43:17.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.416+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01a2d00 con 0x7faff0100c30 2024-06-02T15:43:17.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.416+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0018960 con 0x7faff0100c30 2024-06-02T15:43:17.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.416+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a31a0 con 0x7faff0100c30 2024-06-02T15:43:17.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.417+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff01a3400 con 0x7faff0100c30 2024-06-02T15:43:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.420+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fafe001f020 con 0x7faff0100c30 2024-06-02T15:43:17.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.421+0000 7fafe6a00640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 0x7fafd40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.421+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fafe00e2fc0 con 0x7faff0100c30 2024-06-02T15:43:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.421+0000 7fafed400640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 0x7fafd40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.422+0000 7fafed400640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 0x7fafd40b4cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faff019c300 tx=0x7fafdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:17.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.424+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fafe0026070 con 0x7faff0100c30 2024-06-02T15:43:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:17 smithi120 ceph-mon[36451]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:17 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T15:43:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:17 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:43:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:17 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1952616564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:17 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2034074655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:43:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:17 smithi135 ceph-mon[36839]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:17 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T15:43:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:17 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:43:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:17 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1952616564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:17 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2034074655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:43:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:17 smithi090 ceph-mon[31595]: pgmap v479: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-06-02T15:43:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:17 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T15:43:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:17 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:43:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:17 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1952616564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:17 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2034074655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:43:17.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.630+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faff01a35d0 con 0x7faff0100c30 2024-06-02T15:43:17.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.630+0000 7fafe6a00640 1 -- 172.21.15.90:0/1430693631 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fafe00aa8d0 con 0x7faff0100c30 2024-06-02T15:43:17.628 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-06-02T15:43:17.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 msgr2=0x7fafd40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 0x7fafd40b4cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faff019c300 tx=0x7fafdc005f90 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 msgr2=0x7faff019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafe000e9c0 tx=0x7fafe000ee80 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 shutdown_connections 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fafd40b27f0 0x7fafd40b4cb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faff01045c0 0x7faff01a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faff0103c00 0x7faff019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 --2- 172.21.15.90:0/1430693631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faff0100c30 0x7faff019adf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.633+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 >> 172.21.15.90:0/1430693631 conn(0x7faff00fc580 msgr2=0x7faff01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.634+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 shutdown_connections 2024-06-02T15:43:17.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.634+0000 7fafef200640 1 -- 172.21.15.90:0/1430693631 wait complete. 2024-06-02T15:43:17.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-02T15:43:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 -- 172.21.15.90:0/2643323898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 msgr2=0x7f9878104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 --2- 172.21.15.90:0/2643323898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f9878104640 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f986c009900 tx=0x7f986c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 -- 172.21.15.90:0/2643323898 shutdown_connections 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 --2- 172.21.15.90:0/2643323898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878107590 0x7f9878109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 --2- 172.21.15.90:0/2643323898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878104b80 0x7f9878106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 --2- 172.21.15.90:0/2643323898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f9878104640 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 -- 172.21.15.90:0/2643323898 >> 172.21.15.90:0/2643323898 conn(0x7f98780fc250 msgr2=0x7f98780fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 -- 172.21.15.90:0/2643323898 shutdown_connections 2024-06-02T15:43:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.865+0000 7f987fe00640 1 -- 172.21.15.90:0/2643323898 wait complete. 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.866+0000 7f987fe00640 1 Processor -- start 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.866+0000 7f987fe00640 1 -- start start 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f987819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878107590 0x7f98781a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f987810d4a0 con 0x7f9878107590 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987810d320 con 0x7f9878102250 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f987810d620 con 0x7f9878104b80 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878107590 0x7f98781a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f987819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55420/0 (socket says 172.21.15.90:55420) 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 -- 172.21.15.90:0/601384147 learned_addr learned my addr 172.21.15.90:0/601384147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 -- 172.21.15.90:0/601384147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 msgr2=0x7f987819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f987819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 -- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878107590 msgr2=0x7f98781a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878107590 0x7f98781a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987ca00640 1 -- 172.21.15.90:0/601384147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98781a2b50 con 0x7f9878104b80 2024-06-02T15:43:17.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.867+0000 7f987d400640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f987819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f987ca00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f986400e9f0 tx=0x7f986400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986400cd80 con 0x7f9878104b80 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98640108e0 con 0x7f9878104b80 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864018980 con 0x7f9878104b80 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98781a2e40 con 0x7f9878104b80 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.868+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98781a32b0 con 0x7f9878104b80 2024-06-02T15:43:17.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.869+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98781a34e0 con 0x7f9878104b80 2024-06-02T15:43:17.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.872+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9864010430 con 0x7f9878104b80 2024-06-02T15:43:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.873+0000 7f9876000640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 0x7f98600b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.873+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f98640e2d70 con 0x7f9878104b80 2024-06-02T15:43:17.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.874+0000 7f987d400640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 0x7f98600b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:17.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.874+0000 7f987d400640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 0x7f98600b4d30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f986c009230 tx=0x7f986c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:17.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:17.875+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98640aa030 con 0x7f9878104b80 2024-06-02T15:43:18.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.084+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98781a36b0 con 0x7f9878104b80 2024-06-02T15:43:18.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.085+0000 7f9876000640 1 -- 172.21.15.90:0/601384147 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f98640ae330 con 0x7f9878104b80 2024-06-02T15:43:18.082 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-06-02T15:43:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 msgr2=0x7f98600b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 0x7f98600b4d30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f986c009230 tx=0x7f986c0314a0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 msgr2=0x7f987819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f986400e9f0 tx=0x7f986400eeb0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 shutdown_connections 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f98600b2870 0x7f98600b4d30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9878107590 0x7f98781a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9878104b80 0x7f987819b470 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 --2- 172.21.15.90:0/601384147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9878102250 0x7f987819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 >> 172.21.15.90:0/601384147 conn(0x7f98780fc250 msgr2=0x7f9878101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 shutdown_connections 2024-06-02T15:43:18.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.088+0000 7f987fe00640 1 -- 172.21.15.90:0/601384147 wait complete. 2024-06-02T15:43:18.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 -- 172.21.15.90:0/2074546546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5580754b0 msgr2=0x7fb558075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 --2- 172.21.15.90:0/2074546546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5580754b0 0x7fb558075930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb544009900 tx=0x7fb54402a3f0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 -- 172.21.15.90:0/2074546546 shutdown_connections 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 --2- 172.21.15.90:0/2074546546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb55810ef30 0x7fb558111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 --2- 172.21.15.90:0/2074546546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5580754b0 0x7fb558075930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 --2- 172.21.15.90:0/2074546546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb558076a90 0x7fb558074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 -- 172.21.15.90:0/2074546546 >> 172.21.15.90:0/2074546546 conn(0x7fb5580fdef0 msgr2=0x7fb558100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:18.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 -- 172.21.15.90:0/2074546546 shutdown_connections 2024-06-02T15:43:18.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.314+0000 7fb55d200640 1 -- 172.21.15.90:0/2074546546 wait complete. 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 Processor -- start 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 -- start start 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5580754b0 0x7fb55819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb558076a90 0x7fb55819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:18.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb558114a90 con 0x7fb55810ef30 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb558114910 con 0x7fb5580754b0 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.316+0000 7fb55d200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb558114c10 con 0x7fb558076a90 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47814/0 (socket says 172.21.15.90:47814) 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 -- 172.21.15.90:0/2120134964 learned_addr learned my addr 172.21.15.90:0/2120134964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557400640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5580754b0 0x7fb55819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb556a00640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb558076a90 0x7fb55819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 -- 172.21.15.90:0/2120134964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb558076a90 msgr2=0x7fb55819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb558076a90 0x7fb55819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 -- 172.21.15.90:0/2120134964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5580754b0 msgr2=0x7fb55819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5580754b0 0x7fb55819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 -- 172.21.15.90:0/2120134964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5581a6da0 con 0x7fb55810ef30 2024-06-02T15:43:18.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.317+0000 7fb557e00640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb55000eb10 tx=0x7fb55000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:18.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.318+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550004b20 con 0x7fb55810ef30 2024-06-02T15:43:18.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.318+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb550004cc0 con 0x7fb55810ef30 2024-06-02T15:43:18.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.318+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5581a7090 con 0x7fb55810ef30 2024-06-02T15:43:18.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.318+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550010670 con 0x7fb55810ef30 2024-06-02T15:43:18.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.318+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a7600 con 0x7fb55810ef30 2024-06-02T15:43:18.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.319+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb558105100 con 0x7fb55810ef30 2024-06-02T15:43:18.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.321+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb550005020 con 0x7fb55810ef30 2024-06-02T15:43:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.323+0000 7fb54fe00640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 0x7fb53c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.323+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb5500e2ad0 con 0x7fb55810ef30 2024-06-02T15:43:18.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.324+0000 7fb557400640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 0x7fb53c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:18.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.324+0000 7fb557400640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 0x7fb53c0b4d30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb548009bd0 tx=0x7fb548005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:18.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.327+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5500aa3e0 con 0x7fb55810ef30 2024-06-02T15:43:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:18 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1430693631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:18 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/601384147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.537+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb558063650 con 0x7fb55810ef30 2024-06-02T15:43:18.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.540+0000 7fb54fe00640 1 -- 172.21.15.90:0/2120134964 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fb5500a9ce0 con 0x7fb55810ef30 2024-06-02T15:43:18.538 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-06-02T15:43:18.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 msgr2=0x7fb53c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 0x7fb53c0b4d30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb548009bd0 tx=0x7fb548005f90 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 msgr2=0x7fb5581a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb55000eb10 tx=0x7fb55000c2d0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 shutdown_connections 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb53c0b2870 0x7fb53c0b4d30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55810ef30 0x7fb5581a6730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb558076a90 0x7fb55819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 --2- 172.21.15.90:0/2120134964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5580754b0 0x7fb55819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.543+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 >> 172.21.15.90:0/2120134964 conn(0x7fb5580fdef0 msgr2=0x7fb5581022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.544+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 shutdown_connections 2024-06-02T15:43:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:18.544+0000 7fb55d200640 1 -- 172.21.15.90:0/2120134964 wait complete. 2024-06-02T15:43:18.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:18 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1430693631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:18 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/601384147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.560 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-02T15:43:18.560 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:43:18.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:18 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1430693631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:18 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/601384147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:43:18.997 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:19.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:19 smithi090 ceph-mon[31595]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:19.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:19 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2120134964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:43:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:19 smithi120 ceph-mon[36451]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:19 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2120134964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:43:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:19 smithi135 ceph-mon[36839]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:19 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2120134964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 -- 172.21.15.90:0/2973142443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d0109a80 msgr2=0x7fb3d0109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 --2- 172.21.15.90:0/2973142443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d0109a80 0x7fb3d0109f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009990 tx=0x7fb3bc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 -- 172.21.15.90:0/2973142443 shutdown_connections 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 --2- 172.21.15.90:0/2973142443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010a440 0x7fb3d01168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 --2- 172.21.15.90:0/2973142443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d010e950 0x7fb3d010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 --2- 172.21.15.90:0/2973142443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d0109a80 0x7fb3d0109f00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 -- 172.21.15.90:0/2973142443 >> 172.21.15.90:0/2973142443 conn(0x7fb3d00792d0 msgr2=0x7fb3d0108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 -- 172.21.15.90:0/2973142443 shutdown_connections 2024-06-02T15:43:21.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.295+0000 7fb3d6200700 1 -- 172.21.15.90:0/2973142443 wait complete. 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.296+0000 7fb3d6200700 1 Processor -- start 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.296+0000 7fb3d6200700 1 -- start start 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 0x7fb3d01a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d010a440 0x7fb3d01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0119c80 con 0x7fb3d010e950 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0119b00 con 0x7fb3d0109a80 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d6200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0119e00 con 0x7fb3d010a440 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d5800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d5800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34648/0 (socket says 172.21.15.90:34648) 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d5800700 1 -- 172.21.15.90:0/4286954405 learned_addr learned my addr 172.21.15.90:0/4286954405 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:21.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3d4e00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 0x7fb3d01a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.297+0000 7fb3cfe00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d010a440 0x7fb3d01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 -- 172.21.15.90:0/4286954405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d010a440 msgr2=0x7fb3d01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d010a440 0x7fb3d01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 -- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 msgr2=0x7fb3d01a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 -- 172.21.15.90:0/4286954405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d01a5180 con 0x7fb3d0109a80 2024-06-02T15:43:21.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d5800700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d4e00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 0x7fb3d01a0170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9f0 tx=0x7fb3c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400c9c0 con 0x7fb3d0109a80 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c400cb60 con 0x7fb3d0109a80 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4018680 con 0x7fb3d0109a80 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.298+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d01a5470 con 0x7fb3d0109a80 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.299+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d01accf0 con 0x7fb3d0109a80 2024-06-02T15:43:21.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.300+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d006a230 con 0x7fb3d0109a80 2024-06-02T15:43:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.302+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb3c404aab0 con 0x7fb3d0109a80 2024-06-02T15:43:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.303+0000 7fb3cd600700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 0x7fb3b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.303+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb3c40e2810 con 0x7fb3d0109a80 2024-06-02T15:43:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.304+0000 7fb3cfe00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 0x7fb3b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.304+0000 7fb3cfe00700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 0x7fb3b80b4dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb3d01095e0 tx=0x7fb3bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.306+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3c40aa120 con 0x7fb3d0109a80 2024-06-02T15:43:21.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:21 smithi090 ceph-mon[31595]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:21 smithi120 ceph-mon[36451]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:21 smithi135 ceph-mon[36839]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:21.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.587+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3d002e560 con 0x7fb3d0109a80 2024-06-02T15:43:21.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.588+0000 7fb3cd600700 1 -- 172.21.15.90:0/4286954405 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fb3c40b21a0 con 0x7fb3d0109a80 2024-06-02T15:43:21.586 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:21.586 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:43:15.667079+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:43:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.591+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 msgr2=0x7fb3b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.591+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 0x7fb3b80b4dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb3d01095e0 tx=0x7fb3bc005fd0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.591+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 msgr2=0x7fb3d01a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:21.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.591+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 0x7fb3d01a0170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9f0 tx=0x7fb3c400ed00 comp rx=0 tx=0).stop 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 shutdown_connections 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb3b80b2900 0x7fb3b80b4dc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3d010e950 0x7fb3d01a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3d0109a80 0x7fb3d01a0170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 --2- 172.21.15.90:0/4286954405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3d010a440 0x7fb3d01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 >> 172.21.15.90:0/4286954405 conn(0x7fb3d00792d0 msgr2=0x7fb3d0079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 shutdown_connections 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:21.592+0000 7fb3d6200700 1 -- 172.21.15.90:0/4286954405 wait complete. 2024-06-02T15:43:21.590 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-06-02T15:43:22.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:22 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/4286954405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:22.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:22 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/4286954405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:22.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:22 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/4286954405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:22.584 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-02T15:43:22.584 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-02T15:43:22.584 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T15:43:22.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 -- 172.21.15.90:0/379957980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8104cb0 msgr2=0x7fe2d81050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 --2- 172.21.15.90:0/379957980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8104cb0 0x7fe2d81050d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009900 tx=0x7fe2c802a3f0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 -- 172.21.15.90:0/379957980 shutdown_connections 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 --2- 172.21.15.90:0/379957980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8106860 0x7fe2d810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 --2- 172.21.15.90:0/379957980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 0x7fe2d8106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 --2- 172.21.15.90:0/379957980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8104cb0 0x7fe2d81050d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 -- 172.21.15.90:0/379957980 >> 172.21.15.90:0/379957980 conn(0x7fe2d8100240 msgr2=0x7fe2d8102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 -- 172.21.15.90:0/379957980 shutdown_connections 2024-06-02T15:43:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.779+0000 7fe2dca00640 1 -- 172.21.15.90:0/379957980 wait complete. 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.780+0000 7fe2dca00640 1 Processor -- start 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.780+0000 7fe2dca00640 1 -- start start 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 0x7fe2d819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 0x7fe2d81a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d81107d0 con 0x7fe2d8106860 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8110650 con 0x7fe2d8104cb0 2024-06-02T15:43:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2dca00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d8110950 con 0x7fe2d8105ea0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47020/0 (socket says 172.21.15.90:47020) 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 -- 172.21.15.90:0/1308325942 learned_addr learned my addr 172.21.15.90:0/1308325942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5400640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 0x7fe2d819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d6800640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 0x7fe2d81a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 -- 172.21.15.90:0/1308325942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 msgr2=0x7fe2d819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 0x7fe2d819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 -- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 msgr2=0x7fe2d81a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 0x7fe2d81a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.781+0000 7fe2d5e00640 1 -- 172.21.15.90:0/1308325942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d81a2b80 con 0x7fe2d8104cb0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2d6800640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 0x7fe2d81a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2d5e00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8006010 tx=0x7fe2c80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c803f030 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d81a2e10 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c8031910 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8035070 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.782+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d81a3220 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.783+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d810c0e0 con 0x7fe2d8104cb0 2024-06-02T15:43:22.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.785+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe2c8042030 con 0x7fe2d8104cb0 2024-06-02T15:43:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.787+0000 7fe2cea00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 0x7fe2bc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.787+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe2c8102bc0 con 0x7fe2d8104cb0 2024-06-02T15:43:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.787+0000 7fe2d5400640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 0x7fe2bc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:22.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.788+0000 7fe2d5400640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 0x7fe2bc0b4cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe2d819c470 tx=0x7fe2c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:22.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.791+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2c80ca4d0 con 0x7fe2d8104cb0 2024-06-02T15:43:22.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:22.999+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2d8062e90 con 0x7fe2bc0b27f0 2024-06-02T15:43:23.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.022+0000 7fe2cea00640 1 -- 172.21.15.90:0/1308325942 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7fe2d8062e90 con 0x7fe2bc0b27f0 2024-06-02T15:43:23.026 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:23.028 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 msgr2=0x7fe2bc0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 0x7fe2bc0b4cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe2d819c470 tx=0x7fe2c4005f90 comp rx=0 tx=0).stop 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 msgr2=0x7fe2d819af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8006010 tx=0x7fe2c80042f0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 shutdown_connections 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2bc0b27f0 0x7fe2bc0b4cb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d8106860 0x7fe2d81a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d8105ea0 0x7fe2d819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 --2- 172.21.15.90:0/1308325942 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d8104cb0 0x7fe2d819af60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.028+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 >> 172.21.15.90:0/1308325942 conn(0x7fe2d8100240 msgr2=0x7fe2d81092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:23.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.029+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 shutdown_connections 2024-06-02T15:43:23.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.029+0000 7fe2dca00640 1 -- 172.21.15.90:0/1308325942 wait complete. 2024-06-02T15:43:23.047 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-06-02T15:43:21.825564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665264,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1441,"num_read_kb":820,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375836,"kb_used_data":14204,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565988,"statfs":{"total":1151940427776,"available":1151555571712,"internally_reserved":0,"allocated":14544896,"data_stored":8189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5679,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006572"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706882+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:41:27.706882+0000","last_peered":"2024-06-02T15:41:27.706882+0000","last_clean":"2024-06-02T15:41:27.706882+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:41:27.706882+0000","last_undegraded":"2024-06-02T15:41:27.706882+0000","last_fullsized":"2024-06-02T15:41:27.706882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706280+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:41:27.706280+0000","last_peered":"2024-06-02T15:41:27.706280+0000","last_clean":"2024-06-02T15:41:27.706280+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:41:27.706280+0000","last_undegraded":"2024-06-02T15:41:27.706280+0000","last_fullsized":"2024-06-02T15:41:27.706280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706538+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:41:27.706538+0000","last_peered":"2024-06-02T15:41:27.706538+0000","last_clean":"2024-06-02T15:41:27.706538+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:41:27.706538+0000","last_undegraded":"2024-06-02T15:41:27.706538+0000","last_fullsized":"2024-06-02T15:41:27.706538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692190+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:41:27.692190+0000","last_peered":"2024-06-02T15:41:27.692190+0000","last_clean":"2024-06-02T15:41:27.692190+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:41:27.692190+0000","last_undegraded":"2024-06-02T15:41:27.692190+0000","last_fullsized":"2024-06-02T15:41:27.692190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699200+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:41:27.699200+0000","last_peered":"2024-06-02T15:41:27.699200+0000","last_clean":"2024-06-02T15:41:27.699200+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:41:27.699200+0000","last_undegraded":"2024-06-02T15:41:27.699200+0000","last_fullsized":"2024-06-02T15:41:27.699200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697522+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:41:27.697522+0000","last_peered":"2024-06-02T15:41:27.697522+0000","last_clean":"2024-06-02T15:41:27.697522+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:41:27.697522+0000","last_undegraded":"2024-06-02T15:41:27.697522+0000","last_fullsized":"2024-06-02T15:41:27.697522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.700721+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:41:27.700721+0000","last_peered":"2024-06-02T15:41:27.700721+0000","last_clean":"2024-06-02T15:41:27.700721+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:41:27.700721+0000","last_undegraded":"2024-06-02T15:41:27.700721+0000","last_fullsized":"2024-06-02T15:41:27.700721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711556+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:41:27.711556+0000","last_peered":"2024-06-02T15:41:27.711556+0000","last_clean":"2024-06-02T15:41:27.711556+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:41:27.711556+0000","last_undegraded":"2024-06-02T15:41:27.711556+0000","last_fullsized":"2024-06-02T15:41:27.711556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692972+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:41:27.692972+0000","last_peered":"2024-06-02T15:41:27.692972+0000","last_clean":"2024-06-02T15:41:27.692972+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:41:27.692972+0000","last_undegraded":"2024-06-02T15:41:27.692972+0000","last_fullsized":"2024-06-02T15:41:27.692972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698817+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:41:27.698817+0000","last_peered":"2024-06-02T15:41:27.698817+0000","last_clean":"2024-06-02T15:41:27.698817+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:41:27.698817+0000","last_undegraded":"2024-06-02T15:41:27.698817+0000","last_fullsized":"2024-06-02T15:41:27.698817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695499+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:41:27.695499+0000","last_peered":"2024-06-02T15:41:27.695499+0000","last_clean":"2024-06-02T15:41:27.695499+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:41:27.695499+0000","last_undegraded":"2024-06-02T15:41:27.695499+0000","last_fullsized":"2024-06-02T15:41:27.695499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690188+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:41:27.690188+0000","last_peered":"2024-06-02T15:41:27.690188+0000","last_clean":"2024-06-02T15:41:27.690188+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:41:27.690188+0000","last_undegraded":"2024-06-02T15:41:27.690188+0000","last_fullsized":"2024-06-02T15:41:27.690188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696742+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:41:27.696742+0000","last_peered":"2024-06-02T15:41:27.696742+0000","last_clean":"2024-06-02T15:41:27.696742+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:41:27.696742+0000","last_undegraded":"2024-06-02T15:41:27.696742+0000","last_fullsized":"2024-06-02T15:41:27.696742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699476+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:41:27.699476+0000","last_peered":"2024-06-02T15:41:27.699476+0000","last_clean":"2024-06-02T15:41:27.699476+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:41:27.699476+0000","last_undegraded":"2024-06-02T15:41:27.699476+0000","last_fullsized":"2024-06-02T15:41:27.699476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706081+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:41:27.706081+0000","last_peered":"2024-06-02T15:41:27.706081+0000","last_clean":"2024-06-02T15:41:27.706081+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:41:27.706081+0000","last_undegraded":"2024-06-02T15:41:27.706081+0000","last_fullsized":"2024-06-02T15:41:27.706081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702973+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:41:27.702973+0000","last_peered":"2024-06-02T15:41:27.702973+0000","last_clean":"2024-06-02T15:41:27.702973+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:41:27.702973+0000","last_undegraded":"2024-06-02T15:41:27.702973+0000","last_fullsized":"2024-06-02T15:41:27.702973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706900+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:41:27.706900+0000","last_peered":"2024-06-02T15:41:27.706900+0000","last_clean":"2024-06-02T15:41:27.706900+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:41:27.706900+0000","last_undegraded":"2024-06-02T15:41:27.706900+0000","last_fullsized":"2024-06-02T15:41:27.706900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702270+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:41:27.702270+0000","last_peered":"2024-06-02T15:41:27.702270+0000","last_clean":"2024-06-02T15:41:27.702270+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:41:27.702270+0000","last_undegraded":"2024-06-02T15:41:27.702270+0000","last_fullsized":"2024-06-02T15:41:27.702270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690254+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:41:27.690254+0000","last_peered":"2024-06-02T15:41:27.690254+0000","last_clean":"2024-06-02T15:41:27.690254+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:41:27.690254+0000","last_undegraded":"2024-06-02T15:41:27.690254+0000","last_fullsized":"2024-06-02T15:41:27.690254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:55.648352+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:55.648352+0000","last_peered":"2024-06-02T15:40:55.648352+0000","last_clean":"2024-06-02T15:40:55.648352+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:55.648352+0000","last_undegraded":"2024-06-02T15:40:55.648352+0000","last_fullsized":"2024-06-02T15:40:55.648352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849695+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:20.849695+0000","last_peered":"2024-06-02T15:40:20.849695+0000","last_clean":"2024-06-02T15:40:20.849695+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:20.849695+0000","last_undegraded":"2024-06-02T15:40:20.849695+0000","last_fullsized":"2024-06-02T15:40:20.849695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705339+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:41:27.705339+0000","last_peered":"2024-06-02T15:41:27.705339+0000","last_clean":"2024-06-02T15:41:27.705339+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:41:27.705339+0000","last_undegraded":"2024-06-02T15:41:27.705339+0000","last_fullsized":"2024-06-02T15:41:27.705339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.145879+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:54.145879+0000","last_peered":"2024-06-02T15:40:54.145879+0000","last_clean":"2024-06-02T15:40:54.145879+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:54.145879+0000","last_undegraded":"2024-06-02T15:40:54.145879+0000","last_fullsized":"2024-06-02T15:40:54.145879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:12.337384+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:12.337384+0000","last_peered":"2024-06-02T15:40:12.337384+0000","last_clean":"2024-06-02T15:40:12.337384+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:12.337384+0000","last_undegraded":"2024-06-02T15:40:12.337384+0000","last_fullsized":"2024-06-02T15:40:12.337384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708471+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:41:27.708471+0000","last_peered":"2024-06-02T15:41:27.708471+0000","last_clean":"2024-06-02T15:41:27.708471+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:41:27.708471+0000","last_undegraded":"2024-06-02T15:41:27.708471+0000","last_fullsized":"2024-06-02T15:41:27.708471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031438+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031438+0000","last_peered":"2024-06-02T15:40:00.031438+0000","last_clean":"2024-06-02T15:40:00.031438+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031438+0000","last_undegraded":"2024-06-02T15:40:00.031438+0000","last_fullsized":"2024-06-02T15:40:00.031438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850859+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:45.850859+0000","last_peered":"2024-06-02T15:40:45.850859+0000","last_clean":"2024-06-02T15:40:45.850859+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:45.850859+0000","last_undegraded":"2024-06-02T15:40:45.850859+0000","last_fullsized":"2024-06-02T15:40:45.850859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697673+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:41:27.697673+0000","last_peered":"2024-06-02T15:41:27.697673+0000","last_clean":"2024-06-02T15:41:27.697673+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:41:27.697673+0000","last_undegraded":"2024-06-02T15:41:27.697673+0000","last_fullsized":"2024-06-02T15:41:27.697673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031342+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031342+0000","last_peered":"2024-06-02T15:40:00.031342+0000","last_clean":"2024-06-02T15:40:00.031342+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031342+0000","last_undegraded":"2024-06-02T15:40:00.031342+0000","last_fullsized":"2024-06-02T15:40:00.031342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850848+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:45.850848+0000","last_peered":"2024-06-02T15:40:45.850848+0000","last_clean":"2024-06-02T15:40:45.850848+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:45.850848+0000","last_undegraded":"2024-06-02T15:40:45.850848+0000","last_fullsized":"2024-06-02T15:40:45.850848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696163+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:41:27.696163+0000","last_peered":"2024-06-02T15:41:27.696163+0000","last_clean":"2024-06-02T15:41:27.696163+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:41:27.696163+0000","last_undegraded":"2024-06-02T15:41:27.696163+0000","last_fullsized":"2024-06-02T15:41:27.696163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031260+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031260+0000","last_peered":"2024-06-02T15:40:00.031260+0000","last_clean":"2024-06-02T15:40:00.031260+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031260+0000","last_undegraded":"2024-06-02T15:40:00.031260+0000","last_fullsized":"2024-06-02T15:40:00.031260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":280,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.466055+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:43:17.466055+0000","last_peered":"2024-06-02T15:43:17.466055+0000","last_clean":"2024-06-02T15:43:17.466055+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:43:17.466055+0000","last_undegraded":"2024-06-02T15:43:17.466055+0000","last_fullsized":"2024-06-02T15:43:17.466055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":227,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697597+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:41:27.697597+0000","last_peered":"2024-06-02T15:41:27.697597+0000","last_clean":"2024-06-02T15:41:27.697597+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:41:27.697597+0000","last_undegraded":"2024-06-02T15:41:27.697597+0000","last_fullsized":"2024-06-02T15:41:27.697597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031151+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031151+0000","last_peered":"2024-06-02T15:40:00.031151+0000","last_clean":"2024-06-02T15:40:00.031151+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031151+0000","last_undegraded":"2024-06-02T15:40:00.031151+0000","last_fullsized":"2024-06-02T15:40:00.031151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:42.361651+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:42.361651+0000","last_peered":"2024-06-02T15:40:42.361651+0000","last_clean":"2024-06-02T15:40:42.361651+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:42.361651+0000","last_undegraded":"2024-06-02T15:40:42.361651+0000","last_fullsized":"2024-06-02T15:40:42.361651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711391+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:41:27.711391+0000","last_peered":"2024-06-02T15:41:27.711391+0000","last_clean":"2024-06-02T15:41:27.711391+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:41:27.711391+0000","last_undegraded":"2024-06-02T15:41:27.711391+0000","last_fullsized":"2024-06-02T15:41:27.711391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581825+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:34.581825+0000","last_peered":"2024-06-02T15:40:34.581825+0000","last_clean":"2024-06-02T15:40:34.581825+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:34.581825+0000","last_undegraded":"2024-06-02T15:40:34.581825+0000","last_fullsized":"2024-06-02T15:40:34.581825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463524+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:39.463524+0000","last_peered":"2024-06-02T15:40:39.463524+0000","last_clean":"2024-06-02T15:40:39.463524+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:39.463524+0000","last_undegraded":"2024-06-02T15:40:39.463524+0000","last_fullsized":"2024-06-02T15:40:39.463524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711431+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:41:27.711431+0000","last_peered":"2024-06-02T15:41:27.711431+0000","last_clean":"2024-06-02T15:41:27.711431+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:41:27.711431+0000","last_undegraded":"2024-06-02T15:41:27.711431+0000","last_fullsized":"2024-06-02T15:41:27.711431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:31.928337+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:31.928337+0000","last_peered":"2024-06-02T15:40:31.928337+0000","last_clean":"2024-06-02T15:40:31.928337+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:31.928337+0000","last_undegraded":"2024-06-02T15:40:31.928337+0000","last_fullsized":"2024-06-02T15:40:31.928337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929241+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:38.929241+0000","last_peered":"2024-06-02T15:40:38.929241+0000","last_clean":"2024-06-02T15:40:38.929241+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:38.929241+0000","last_undegraded":"2024-06-02T15:40:38.929241+0000","last_fullsized":"2024-06-02T15:40:38.929241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710382+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:41:27.710382+0000","last_peered":"2024-06-02T15:41:27.710382+0000","last_clean":"2024-06-02T15:41:27.710382+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:41:27.710382+0000","last_undegraded":"2024-06-02T15:41:27.710382+0000","last_fullsized":"2024-06-02T15:41:27.710382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.446626+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:50.446626+0000","last_peered":"2024-06-02T15:40:50.446626+0000","last_clean":"2024-06-02T15:40:50.446626+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:50.446626+0000","last_undegraded":"2024-06-02T15:40:50.446626+0000","last_fullsized":"2024-06-02T15:40:50.446626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":271,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:15.462318+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:43:15.462318+0000","last_peered":"2024-06-02T15:43:15.462318+0000","last_clean":"2024-06-02T15:43:15.462318+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:43:15.462318+0000","last_undegraded":"2024-06-02T15:43:15.462318+0000","last_fullsized":"2024-06-02T15:43:15.462318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":221,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701105+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:41:27.701105+0000","last_peered":"2024-06-02T15:41:27.701105+0000","last_clean":"2024-06-02T15:41:27.701105+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:41:27.701105+0000","last_undegraded":"2024-06-02T15:41:27.701105+0000","last_fullsized":"2024-06-02T15:41:27.701105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854789+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:50.854789+0000","last_peered":"2024-06-02T15:40:50.854789+0000","last_clean":"2024-06-02T15:40:50.854789+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:50.854789+0000","last_undegraded":"2024-06-02T15:40:50.854789+0000","last_fullsized":"2024-06-02T15:40:50.854789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:40.421522+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:40.421522+0000","last_peered":"2024-06-02T15:40:40.421522+0000","last_clean":"2024-06-02T15:40:40.421522+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:40.421522+0000","last_undegraded":"2024-06-02T15:40:40.421522+0000","last_fullsized":"2024-06-02T15:40:40.421522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706943+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:41:27.706943+0000","last_peered":"2024-06-02T15:41:27.706943+0000","last_clean":"2024-06-02T15:41:27.706943+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:41:27.706943+0000","last_undegraded":"2024-06-02T15:41:27.706943+0000","last_fullsized":"2024-06-02T15:41:27.706943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.145181+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:49.145181+0000","last_peered":"2024-06-02T15:40:49.145181+0000","last_clean":"2024-06-02T15:40:49.145181+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:49.145181+0000","last_undegraded":"2024-06-02T15:40:49.145181+0000","last_fullsized":"2024-06-02T15:40:49.145181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929293+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:38.929293+0000","last_peered":"2024-06-02T15:40:38.929293+0000","last_clean":"2024-06-02T15:40:38.929293+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:38.929293+0000","last_undegraded":"2024-06-02T15:40:38.929293+0000","last_fullsized":"2024-06-02T15:40:38.929293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698278+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:41:27.698278+0000","last_peered":"2024-06-02T15:41:27.698278+0000","last_clean":"2024-06-02T15:41:27.698278+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:41:27.698278+0000","last_undegraded":"2024-06-02T15:41:27.698278+0000","last_fullsized":"2024-06-02T15:41:27.698278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854862+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:50.854862+0000","last_peered":"2024-06-02T15:40:50.854862+0000","last_clean":"2024-06-02T15:40:50.854862+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:50.854862+0000","last_undegraded":"2024-06-02T15:40:50.854862+0000","last_fullsized":"2024-06-02T15:40:50.854862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":555,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:16.464411+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:43:16.464411+0000","last_peered":"2024-06-02T15:43:16.464411+0000","last_clean":"2024-06-02T15:43:16.464411+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:43:16.464411+0000","last_undegraded":"2024-06-02T15:43:16.464411+0000","last_fullsized":"2024-06-02T15:43:16.464411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69988,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":290,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685871+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:41:27.685871+0000","last_peered":"2024-06-02T15:41:27.685871+0000","last_clean":"2024-06-02T15:41:27.685871+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:41:27.685871+0000","last_undegraded":"2024-06-02T15:41:27.685871+0000","last_fullsized":"2024-06-02T15:41:27.685871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.931198+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:53.931198+0000","last_peered":"2024-06-02T15:40:53.931198+0000","last_clean":"2024-06-02T15:40:53.931198+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:53.931198+0000","last_undegraded":"2024-06-02T15:40:53.931198+0000","last_fullsized":"2024-06-02T15:40:53.931198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646104+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:35.646104+0000","last_peered":"2024-06-02T15:40:35.646104+0000","last_clean":"2024-06-02T15:40:35.646104+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:35.646104+0000","last_undegraded":"2024-06-02T15:40:35.646104+0000","last_fullsized":"2024-06-02T15:40:35.646104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711870+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:41:27.711870+0000","last_peered":"2024-06-02T15:41:27.711870+0000","last_clean":"2024-06-02T15:41:27.711870+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:41:27.711870+0000","last_undegraded":"2024-06-02T15:41:27.711870+0000","last_fullsized":"2024-06-02T15:41:27.711870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.583429+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:49.583429+0000","last_peered":"2024-06-02T15:40:49.583429+0000","last_clean":"2024-06-02T15:40:49.583429+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:49.583429+0000","last_undegraded":"2024-06-02T15:40:49.583429+0000","last_fullsized":"2024-06-02T15:40:49.583429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:33.928487+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:33.928487+0000","last_peered":"2024-06-02T15:40:33.928487+0000","last_clean":"2024-06-02T15:40:33.928487+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:33.928487+0000","last_undegraded":"2024-06-02T15:40:33.928487+0000","last_fullsized":"2024-06-02T15:40:33.928487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696588+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:41:27.696588+0000","last_peered":"2024-06-02T15:41:27.696588+0000","last_clean":"2024-06-02T15:41:27.696588+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:41:27.696588+0000","last_undegraded":"2024-06-02T15:41:27.696588+0000","last_fullsized":"2024-06-02T15:41:27.696588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.911547+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:53.911547+0000","last_peered":"2024-06-02T15:40:53.911547+0000","last_clean":"2024-06-02T15:40:53.911547+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:53.911547+0000","last_undegraded":"2024-06-02T15:40:53.911547+0000","last_fullsized":"2024-06-02T15:40:53.911547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":278,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615898+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:43:17.615898+0000","last_peered":"2024-06-02T15:43:17.615898+0000","last_clean":"2024-06-02T15:43:17.615898+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:43:17.615898+0000","last_undegraded":"2024-06-02T15:43:17.615898+0000","last_fullsized":"2024-06-02T15:43:17.615898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687560+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:41:27.687560+0000","last_peered":"2024-06-02T15:41:27.687560+0000","last_clean":"2024-06-02T15:41:27.687560+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:41:27.687560+0000","last_undegraded":"2024-06-02T15:41:27.687560+0000","last_fullsized":"2024-06-02T15:41:27.687560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.466042+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:54.466042+0000","last_peered":"2024-06-02T15:40:54.466042+0000","last_clean":"2024-06-02T15:40:54.466042+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:54.466042+0000","last_undegraded":"2024-06-02T15:40:54.466042+0000","last_fullsized":"2024-06-02T15:40:54.466042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849838+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:35.849838+0000","last_peered":"2024-06-02T15:40:35.849838+0000","last_clean":"2024-06-02T15:40:35.849838+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:35.849838+0000","last_undegraded":"2024-06-02T15:40:35.849838+0000","last_fullsized":"2024-06-02T15:40:35.849838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694458+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:41:27.694458+0000","last_peered":"2024-06-02T15:41:27.694458+0000","last_clean":"2024-06-02T15:41:27.694458+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:41:27.694458+0000","last_undegraded":"2024-06-02T15:41:27.694458+0000","last_fullsized":"2024-06-02T15:41:27.694458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929257+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:38.929257+0000","last_peered":"2024-06-02T15:40:38.929257+0000","last_clean":"2024-06-02T15:40:38.929257+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:38.929257+0000","last_undegraded":"2024-06-02T15:40:38.929257+0000","last_fullsized":"2024-06-02T15:40:38.929257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849304+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:30.849304+0000","last_peered":"2024-06-02T15:40:30.849304+0000","last_clean":"2024-06-02T15:40:30.849304+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:30.849304+0000","last_undegraded":"2024-06-02T15:40:30.849304+0000","last_fullsized":"2024-06-02T15:40:30.849304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695493+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:41:27.695493+0000","last_peered":"2024-06-02T15:41:27.695493+0000","last_clean":"2024-06-02T15:41:27.695493+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:41:27.695493+0000","last_undegraded":"2024-06-02T15:41:27.695493+0000","last_fullsized":"2024-06-02T15:41:27.695493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849864+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:35.849864+0000","last_peered":"2024-06-02T15:40:35.849864+0000","last_clean":"2024-06-02T15:40:35.849864+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:35.849864+0000","last_undegraded":"2024-06-02T15:40:35.849864+0000","last_fullsized":"2024-06-02T15:40:35.849864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849635+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:20.849635+0000","last_peered":"2024-06-02T15:40:20.849635+0000","last_clean":"2024-06-02T15:40:20.849635+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:20.849635+0000","last_undegraded":"2024-06-02T15:40:20.849635+0000","last_fullsized":"2024-06-02T15:40:20.849635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693299+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:41:27.693299+0000","last_peered":"2024-06-02T15:41:27.693299+0000","last_clean":"2024-06-02T15:41:27.693299+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:41:27.693299+0000","last_undegraded":"2024-06-02T15:41:27.693299+0000","last_fullsized":"2024-06-02T15:41:27.693299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.462890+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:34.462890+0000","last_peered":"2024-06-02T15:40:34.462890+0000","last_clean":"2024-06-02T15:40:34.462890+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:34.462890+0000","last_undegraded":"2024-06-02T15:40:34.462890+0000","last_fullsized":"2024-06-02T15:40:34.462890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.419076+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:20.419076+0000","last_peered":"2024-06-02T15:40:20.419076+0000","last_clean":"2024-06-02T15:40:20.419076+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:20.419076+0000","last_undegraded":"2024-06-02T15:40:20.419076+0000","last_fullsized":"2024-06-02T15:40:20.419076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.676935+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:41:27.676935+0000","last_peered":"2024-06-02T15:41:27.676935+0000","last_clean":"2024-06-02T15:41:27.676935+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:41:27.676935+0000","last_undegraded":"2024-06-02T15:41:27.676935+0000","last_fullsized":"2024-06-02T15:41:27.676935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463609+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:39.463609+0000","last_peered":"2024-06-02T15:40:39.463609+0000","last_clean":"2024-06-02T15:40:39.463609+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:39.463609+0000","last_undegraded":"2024-06-02T15:40:39.463609+0000","last_fullsized":"2024-06-02T15:40:39.463609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690130+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:41:27.690130+0000","last_peered":"2024-06-02T15:41:27.690130+0000","last_clean":"2024-06-02T15:41:27.690130+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:41:27.690130+0000","last_undegraded":"2024-06-02T15:41:27.690130+0000","last_fullsized":"2024-06-02T15:41:27.690130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":274,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615195+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:43:17.615195+0000","last_peered":"2024-06-02T15:43:17.615195+0000","last_clean":"2024-06-02T15:43:17.615195+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:43:17.615195+0000","last_undegraded":"2024-06-02T15:43:17.615195+0000","last_fullsized":"2024-06-02T15:43:17.615195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":223,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.420757+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:35.420757+0000","last_peered":"2024-06-02T15:40:35.420757+0000","last_clean":"2024-06-02T15:40:35.420757+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:35.420757+0000","last_undegraded":"2024-06-02T15:40:35.420757+0000","last_fullsized":"2024-06-02T15:40:35.420757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708881+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:41:27.708881+0000","last_peered":"2024-06-02T15:41:27.708881+0000","last_clean":"2024-06-02T15:41:27.708881+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:41:27.708881+0000","last_undegraded":"2024-06-02T15:41:27.708881+0000","last_fullsized":"2024-06-02T15:41:27.708881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:19.142002+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:19.142002+0000","last_peered":"2024-06-02T15:40:19.142002+0000","last_clean":"2024-06-02T15:40:19.142002+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:19.142002+0000","last_undegraded":"2024-06-02T15:40:19.142002+0000","last_fullsized":"2024-06-02T15:40:19.142002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:10.424476+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:41:10.424476+0000","last_peered":"2024-06-02T15:41:10.424476+0000","last_clean":"2024-06-02T15:41:10.424476+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:41:10.424476+0000","last_undegraded":"2024-06-02T15:41:10.424476+0000","last_fullsized":"2024-06-02T15:41:10.424476+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929309+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:38.929309+0000","last_peered":"2024-06-02T15:40:38.929309+0000","last_clean":"2024-06-02T15:40:38.929309+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:38.929309+0000","last_undegraded":"2024-06-02T15:40:38.929309+0000","last_fullsized":"2024-06-02T15:40:38.929309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685503+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:41:27.685503+0000","last_peered":"2024-06-02T15:41:27.685503+0000","last_clean":"2024-06-02T15:41:27.685503+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:41:27.685503+0000","last_undegraded":"2024-06-02T15:41:27.685503+0000","last_fullsized":"2024-06-02T15:41:27.685503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:15.418582+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:15.418582+0000","last_peered":"2024-06-02T15:40:15.418582+0000","last_clean":"2024-06-02T15:40:15.418582+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:15.418582+0000","last_undegraded":"2024-06-02T15:40:15.418582+0000","last_fullsized":"2024-06-02T15:40:15.418582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581781+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:34.581781+0000","last_peered":"2024-06-02T15:40:34.581781+0000","last_clean":"2024-06-02T15:40:34.581781+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:34.581781+0000","last_undegraded":"2024-06-02T15:40:34.581781+0000","last_fullsized":"2024-06-02T15:40:34.581781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.848207+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:20.848207+0000","last_peered":"2024-06-02T15:40:20.848207+0000","last_clean":"2024-06-02T15:40:20.848207+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:20.848207+0000","last_undegraded":"2024-06-02T15:40:20.848207+0000","last_fullsized":"2024-06-02T15:40:20.848207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693951+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:41:27.693951+0000","last_peered":"2024-06-02T15:41:27.693951+0000","last_clean":"2024-06-02T15:41:27.693951+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:41:27.693951+0000","last_undegraded":"2024-06-02T15:41:27.693951+0000","last_fullsized":"2024-06-02T15:41:27.693951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:32.340299+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:32.340299+0000","last_peered":"2024-06-02T15:40:32.340299+0000","last_clean":"2024-06-02T15:40:32.340299+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:32.340299+0000","last_undegraded":"2024-06-02T15:40:32.340299+0000","last_fullsized":"2024-06-02T15:40:32.340299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:42:35.870244+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:42:35.870244+0000","last_peered":"2024-06-02T15:42:35.870244+0000","last_clean":"2024-06-02T15:42:35.870244+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:42:35.870244+0000","last_undegraded":"2024-06-02T15:42:35.870244+0000","last_fullsized":"2024-06-02T15:42:35.870244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.709823+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:41:27.709823+0000","last_peered":"2024-06-02T15:41:27.709823+0000","last_clean":"2024-06-02T15:41:27.709823+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:41:27.709823+0000","last_undegraded":"2024-06-02T15:41:27.709823+0000","last_fullsized":"2024-06-02T15:41:27.709823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646076+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:35.646076+0000","last_peered":"2024-06-02T15:40:35.646076+0000","last_clean":"2024-06-02T15:40:35.646076+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:35.646076+0000","last_undegraded":"2024-06-02T15:40:35.646076+0000","last_fullsized":"2024-06-02T15:40:35.646076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:24.142446+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:24.142446+0000","last_peered":"2024-06-02T15:40:24.142446+0000","last_clean":"2024-06-02T15:40:24.142446+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:24.142446+0000","last_undegraded":"2024-06-02T15:40:24.142446+0000","last_fullsized":"2024-06-02T15:40:24.142446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711979+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:41:27.711979+0000","last_peered":"2024-06-02T15:41:27.711979+0000","last_clean":"2024-06-02T15:41:27.711979+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:41:27.711979+0000","last_undegraded":"2024-06-02T15:41:27.711979+0000","last_fullsized":"2024-06-02T15:41:27.711979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144738+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:44.144738+0000","last_peered":"2024-06-02T15:40:44.144738+0000","last_clean":"2024-06-02T15:40:44.144738+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:44.144738+0000","last_undegraded":"2024-06-02T15:40:44.144738+0000","last_fullsized":"2024-06-02T15:40:44.144738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:26.381402+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:26.381402+0000","last_peered":"2024-06-02T15:40:26.381402+0000","last_clean":"2024-06-02T15:40:26.381402+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:26.381402+0000","last_undegraded":"2024-06-02T15:40:26.381402+0000","last_fullsized":"2024-06-02T15:40:26.381402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.680443+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:41:27.680443+0000","last_peered":"2024-06-02T15:41:27.680443+0000","last_clean":"2024-06-02T15:41:27.680443+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:41:27.680443+0000","last_undegraded":"2024-06-02T15:41:27.680443+0000","last_fullsized":"2024-06-02T15:41:27.680443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.853966+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:45.853966+0000","last_peered":"2024-06-02T15:40:45.853966+0000","last_clean":"2024-06-02T15:40:45.853966+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:45.853966+0000","last_undegraded":"2024-06-02T15:40:45.853966+0000","last_fullsized":"2024-06-02T15:40:45.853966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:22.338983+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:22.338983+0000","last_peered":"2024-06-02T15:40:22.338983+0000","last_clean":"2024-06-02T15:40:22.338983+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:22.338983+0000","last_undegraded":"2024-06-02T15:40:22.338983+0000","last_fullsized":"2024-06-02T15:40:22.338983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703948+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:41:27.703948+0000","last_peered":"2024-06-02T15:41:27.703948+0000","last_clean":"2024-06-02T15:41:27.703948+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:41:27.703948+0000","last_undegraded":"2024-06-02T15:41:27.703948+0000","last_fullsized":"2024-06-02T15:41:27.703948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144717+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:44.144717+0000","last_peered":"2024-06-02T15:40:44.144717+0000","last_clean":"2024-06-02T15:40:44.144717+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:44.144717+0000","last_undegraded":"2024-06-02T15:40:44.144717+0000","last_fullsized":"2024-06-02T15:40:44.144717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:09.585558+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:41:09.585558+0000","last_peered":"2024-06-02T15:41:09.585558+0000","last_clean":"2024-06-02T15:41:09.585558+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:41:09.585558+0000","last_undegraded":"2024-06-02T15:41:09.585558+0000","last_fullsized":"2024-06-02T15:41:09.585558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711497+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:41:27.711497+0000","last_peered":"2024-06-02T15:41:27.711497+0000","last_clean":"2024-06-02T15:41:27.711497+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:41:27.711497+0000","last_undegraded":"2024-06-02T15:41:27.711497+0000","last_fullsized":"2024-06-02T15:41:27.711497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144729+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:44.144729+0000","last_peered":"2024-06-02T15:40:44.144729+0000","last_clean":"2024-06-02T15:40:44.144729+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:44.144729+0000","last_undegraded":"2024-06-02T15:40:44.144729+0000","last_fullsized":"2024-06-02T15:40:44.144729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849271+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:30.849271+0000","last_peered":"2024-06-02T15:40:30.849271+0000","last_clean":"2024-06-02T15:40:30.849271+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:30.849271+0000","last_undegraded":"2024-06-02T15:40:30.849271+0000","last_fullsized":"2024-06-02T15:40:30.849271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692818+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:41:27.692818+0000","last_peered":"2024-06-02T15:41:27.692818+0000","last_clean":"2024-06-02T15:41:27.692818+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:41:27.692818+0000","last_undegraded":"2024-06-02T15:41:27.692818+0000","last_fullsized":"2024-06-02T15:41:27.692818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:41.383274+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:41.383274+0000","last_peered":"2024-06-02T15:40:41.383274+0000","last_clean":"2024-06-02T15:40:41.383274+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:41.383274+0000","last_undegraded":"2024-06-02T15:40:41.383274+0000","last_fullsized":"2024-06-02T15:40:41.383274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.645527+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:30.645527+0000","last_peered":"2024-06-02T15:40:30.645527+0000","last_clean":"2024-06-02T15:40:30.645527+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:30.645527+0000","last_undegraded":"2024-06-02T15:40:30.645527+0000","last_fullsized":"2024-06-02T15:40:30.645527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694463+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:41:27.694463+0000","last_peered":"2024-06-02T15:41:27.694463+0000","last_clean":"2024-06-02T15:41:27.694463+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:41:27.694463+0000","last_undegraded":"2024-06-02T15:41:27.694463+0000","last_fullsized":"2024-06-02T15:41:27.694463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:46.930876+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:46.930876+0000","last_peered":"2024-06-02T15:40:46.930876+0000","last_clean":"2024-06-02T15:40:46.930876+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:46.930876+0000","last_undegraded":"2024-06-02T15:40:46.930876+0000","last_fullsized":"2024-06-02T15:40:46.930876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:28.928032+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:28.928032+0000","last_peered":"2024-06-02T15:40:28.928032+0000","last_clean":"2024-06-02T15:40:28.928032+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:28.928032+0000","last_undegraded":"2024-06-02T15:40:28.928032+0000","last_fullsized":"2024-06-02T15:40:28.928032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705291+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:41:27.705291+0000","last_peered":"2024-06-02T15:41:27.705291+0000","last_clean":"2024-06-02T15:41:27.705291+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:41:27.705291+0000","last_undegraded":"2024-06-02T15:41:27.705291+0000","last_fullsized":"2024-06-02T15:41:27.705291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929210+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:38.929210+0000","last_peered":"2024-06-02T15:40:38.929210+0000","last_clean":"2024-06-02T15:40:38.929210+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:38.929210+0000","last_undegraded":"2024-06-02T15:40:38.929210+0000","last_fullsized":"2024-06-02T15:40:38.929210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:29.462003+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:29.462003+0000","last_peered":"2024-06-02T15:40:29.462003+0000","last_clean":"2024-06-02T15:40:29.462003+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:29.462003+0000","last_undegraded":"2024-06-02T15:40:29.462003+0000","last_fullsized":"2024-06-02T15:40:29.462003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685479+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:41:27.685479+0000","last_peered":"2024-06-02T15:41:27.685479+0000","last_clean":"2024-06-02T15:41:27.685479+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:41:27.685479+0000","last_undegraded":"2024-06-02T15:41:27.685479+0000","last_fullsized":"2024-06-02T15:41:27.685479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687491+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:41:27.687491+0000","last_peered":"2024-06-02T15:41:27.687491+0000","last_clean":"2024-06-02T15:41:27.687491+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:41:27.687491+0000","last_undegraded":"2024-06-02T15:41:27.687491+0000","last_fullsized":"2024-06-02T15:41:27.687491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686399+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:41:27.686399+0000","last_peered":"2024-06-02T15:41:27.686399+0000","last_clean":"2024-06-02T15:41:27.686399+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:41:27.686399+0000","last_undegraded":"2024-06-02T15:41:27.686399+0000","last_fullsized":"2024-06-02T15:41:27.686399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701372+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:41:27.701372+0000","last_peered":"2024-06-02T15:41:27.701372+0000","last_clean":"2024-06-02T15:41:27.701372+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:41:27.701372+0000","last_undegraded":"2024-06-02T15:41:27.701372+0000","last_fullsized":"2024-06-02T15:41:27.701372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701977+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:41:27.701977+0000","last_peered":"2024-06-02T15:41:27.701977+0000","last_clean":"2024-06-02T15:41:27.701977+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:41:27.701977+0000","last_undegraded":"2024-06-02T15:41:27.701977+0000","last_fullsized":"2024-06-02T15:41:27.701977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704468+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:41:27.704468+0000","last_peered":"2024-06-02T15:41:27.704468+0000","last_clean":"2024-06-02T15:41:27.704468+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:41:27.704468+0000","last_undegraded":"2024-06-02T15:41:27.704468+0000","last_fullsized":"2024-06-02T15:41:27.704468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686919+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:41:27.686919+0000","last_peered":"2024-06-02T15:41:27.686919+0000","last_clean":"2024-06-02T15:41:27.686919+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:41:27.686919+0000","last_undegraded":"2024-06-02T15:41:27.686919+0000","last_fullsized":"2024-06-02T15:41:27.686919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.691250+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:41:27.691250+0000","last_peered":"2024-06-02T15:41:27.691250+0000","last_clean":"2024-06-02T15:41:27.691250+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:41:27.691250+0000","last_undegraded":"2024-06-02T15:41:27.691250+0000","last_fullsized":"2024-06-02T15:41:27.691250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702170+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:41:27.702170+0000","last_peered":"2024-06-02T15:41:27.702170+0000","last_clean":"2024-06-02T15:41:27.702170+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:41:27.702170+0000","last_undegraded":"2024-06-02T15:41:27.702170+0000","last_fullsized":"2024-06-02T15:41:27.702170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711806+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:41:27.711806+0000","last_peered":"2024-06-02T15:41:27.711806+0000","last_clean":"2024-06-02T15:41:27.711806+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:41:27.711806+0000","last_undegraded":"2024-06-02T15:41:27.711806+0000","last_fullsized":"2024-06-02T15:41:27.711806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706949+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:41:27.706949+0000","last_peered":"2024-06-02T15:41:27.706949+0000","last_clean":"2024-06-02T15:41:27.706949+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:41:27.706949+0000","last_undegraded":"2024-06-02T15:41:27.706949+0000","last_fullsized":"2024-06-02T15:41:27.706949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711917+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:41:27.711917+0000","last_peered":"2024-06-02T15:41:27.711917+0000","last_clean":"2024-06-02T15:41:27.711917+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:41:27.711917+0000","last_undegraded":"2024-06-02T15:41:27.711917+0000","last_fullsized":"2024-06-02T15:41:27.711917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703925+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:41:27.703925+0000","last_peered":"2024-06-02T15:41:27.703925+0000","last_clean":"2024-06-02T15:41:27.703925+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:41:27.703925+0000","last_undegraded":"2024-06-02T15:41:27.703925+0000","last_fullsized":"2024-06-02T15:41:27.703925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698726+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:41:27.698726+0000","last_peered":"2024-06-02T15:41:27.698726+0000","last_clean":"2024-06-02T15:41:27.698726+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:41:27.698726+0000","last_undegraded":"2024-06-02T15:41:27.698726+0000","last_fullsized":"2024-06-02T15:41:27.698726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74662,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1365,"num_read_kb":756,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874323,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1728512,"data_stored":1175948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037850,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430309,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1114316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593835,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":583470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":515364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149819,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953591,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":588742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790065,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1105858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:43:23.057 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T15:43:23.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 -- 172.21.15.90:0/903615750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 msgr2=0x7fd564075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 --2- 172.21.15.90:0/903615750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd564075890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009900 tx=0x7fd54c02a480 comp rx=0 tx=0).stop 2024-06-02T15:43:23.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 -- 172.21.15.90:0/903615750 shutdown_connections 2024-06-02T15:43:23.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 --2- 172.21.15.90:0/903615750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd56410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 --2- 172.21.15.90:0/903615750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd564075890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 --2- 172.21.15.90:0/903615750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 0x7fd564074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 -- 172.21.15.90:0/903615750 >> 172.21.15.90:0/903615750 conn(0x7fd5640fe030 msgr2=0x7fd564100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:23.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 -- 172.21.15.90:0/903615750 shutdown_connections 2024-06-02T15:43:23.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.291+0000 7fd56aa00640 1 -- 172.21.15.90:0/903615750 wait complete. 2024-06-02T15:43:23.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 Processor -- start 2024-06-02T15:43:23.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 -- start start 2024-06-02T15:43:23.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 0x7fd56419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd5641a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd564110830 con 0x7fd564075430 2024-06-02T15:43:23.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641106b0 con 0x7fd564076a10 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.292+0000 7fd56aa00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5641109b0 con 0x7fd56410a950 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34684/0 (socket says 172.21.15.90:34684) 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 -- 172.21.15.90:0/2879307612 learned_addr learned my addr 172.21.15.90:0/2879307612 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd568a00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd5641a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd55b400640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 0x7fd56419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 -- 172.21.15.90:0/2879307612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 msgr2=0x7fd5641a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd5641a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 -- 172.21.15.90:0/2879307612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 msgr2=0x7fd56419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 0x7fd56419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 -- 172.21.15.90:0/2879307612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5641a2ae0 con 0x7fd564075430 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd568a00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd5641a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.293+0000 7fd563e00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd55400ea60 tx=0x7fd55400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.294+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400cde0 con 0x7fd564075430 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.294+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5540035f0 con 0x7fd564075430 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.294+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5641a2dd0 con 0x7fd564075430 2024-06-02T15:43:23.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.294+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554010690 con 0x7fd564075430 2024-06-02T15:43:23.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.294+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a3230 con 0x7fd564075430 2024-06-02T15:43:23.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.295+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd564109710 con 0x7fd564075430 2024-06-02T15:43:23.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.297+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd5540037a0 con 0x7fd564075430 2024-06-02T15:43:23.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.299+0000 7fd561600640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 0x7fd5480b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.299+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd5540e28c0 con 0x7fd564075430 2024-06-02T15:43:23.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.299+0000 7fd55b400640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 0x7fd5480b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.300+0000 7fd55b400640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 0x7fd5480b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd54c031500 tx=0x7fd54c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:23.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.303+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5540aa1d0 con 0x7fd564075430 2024-06-02T15:43:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:23 smithi090 ceph-mon[31595]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-02T15:43:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:23 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:43:23.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.506+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd564062e90 con 0x7fd5480b27f0 2024-06-02T15:43:23.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.517+0000 7fd561600640 1 -- 172.21.15.90:0/2879307612 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7fd564062e90 con 0x7fd5480b27f0 2024-06-02T15:43:23.519 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:23.521 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:43:23.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 msgr2=0x7fd5480b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 0x7fd5480b4cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd54c031500 tx=0x7fd54c005a20 comp rx=0 tx=0).stop 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 msgr2=0x7fd56419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd55400ea60 tx=0x7fd55400ef20 comp rx=0 tx=0).stop 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 shutdown_connections 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd5480b27f0 0x7fd5480b4cb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd56410a950 0x7fd5641a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd564076a10 0x7fd56419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 --2- 172.21.15.90:0/2879307612 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd564075430 0x7fd56419aec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 >> 172.21.15.90:0/2879307612 conn(0x7fd5640fe030 msgr2=0x7fd5641068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:23.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 shutdown_connections 2024-06-02T15:43:23.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.523+0000 7fd56aa00640 1 -- 172.21.15.90:0/2879307612 wait complete. 2024-06-02T15:43:23.544 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-06-02T15:43:21.825564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665264,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1441,"num_read_kb":820,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375836,"kb_used_data":14204,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565988,"statfs":{"total":1151940427776,"available":1151555571712,"internally_reserved":0,"allocated":14544896,"data_stored":8189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5679,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006572"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706882+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:41:27.706882+0000","last_peered":"2024-06-02T15:41:27.706882+0000","last_clean":"2024-06-02T15:41:27.706882+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:41:27.706882+0000","last_undegraded":"2024-06-02T15:41:27.706882+0000","last_fullsized":"2024-06-02T15:41:27.706882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706280+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:41:27.706280+0000","last_peered":"2024-06-02T15:41:27.706280+0000","last_clean":"2024-06-02T15:41:27.706280+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:41:27.706280+0000","last_undegraded":"2024-06-02T15:41:27.706280+0000","last_fullsized":"2024-06-02T15:41:27.706280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706538+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:41:27.706538+0000","last_peered":"2024-06-02T15:41:27.706538+0000","last_clean":"2024-06-02T15:41:27.706538+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:41:27.706538+0000","last_undegraded":"2024-06-02T15:41:27.706538+0000","last_fullsized":"2024-06-02T15:41:27.706538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692190+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:41:27.692190+0000","last_peered":"2024-06-02T15:41:27.692190+0000","last_clean":"2024-06-02T15:41:27.692190+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:41:27.692190+0000","last_undegraded":"2024-06-02T15:41:27.692190+0000","last_fullsized":"2024-06-02T15:41:27.692190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699200+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:41:27.699200+0000","last_peered":"2024-06-02T15:41:27.699200+0000","last_clean":"2024-06-02T15:41:27.699200+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:41:27.699200+0000","last_undegraded":"2024-06-02T15:41:27.699200+0000","last_fullsized":"2024-06-02T15:41:27.699200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697522+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:41:27.697522+0000","last_peered":"2024-06-02T15:41:27.697522+0000","last_clean":"2024-06-02T15:41:27.697522+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:41:27.697522+0000","last_undegraded":"2024-06-02T15:41:27.697522+0000","last_fullsized":"2024-06-02T15:41:27.697522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.700721+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:41:27.700721+0000","last_peered":"2024-06-02T15:41:27.700721+0000","last_clean":"2024-06-02T15:41:27.700721+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:41:27.700721+0000","last_undegraded":"2024-06-02T15:41:27.700721+0000","last_fullsized":"2024-06-02T15:41:27.700721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711556+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:41:27.711556+0000","last_peered":"2024-06-02T15:41:27.711556+0000","last_clean":"2024-06-02T15:41:27.711556+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:41:27.711556+0000","last_undegraded":"2024-06-02T15:41:27.711556+0000","last_fullsized":"2024-06-02T15:41:27.711556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692972+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:41:27.692972+0000","last_peered":"2024-06-02T15:41:27.692972+0000","last_clean":"2024-06-02T15:41:27.692972+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:41:27.692972+0000","last_undegraded":"2024-06-02T15:41:27.692972+0000","last_fullsized":"2024-06-02T15:41:27.692972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698817+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:41:27.698817+0000","last_peered":"2024-06-02T15:41:27.698817+0000","last_clean":"2024-06-02T15:41:27.698817+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:41:27.698817+0000","last_undegraded":"2024-06-02T15:41:27.698817+0000","last_fullsized":"2024-06-02T15:41:27.698817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695499+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:41:27.695499+0000","last_peered":"2024-06-02T15:41:27.695499+0000","last_clean":"2024-06-02T15:41:27.695499+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:41:27.695499+0000","last_undegraded":"2024-06-02T15:41:27.695499+0000","last_fullsized":"2024-06-02T15:41:27.695499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690188+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:41:27.690188+0000","last_peered":"2024-06-02T15:41:27.690188+0000","last_clean":"2024-06-02T15:41:27.690188+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:41:27.690188+0000","last_undegraded":"2024-06-02T15:41:27.690188+0000","last_fullsized":"2024-06-02T15:41:27.690188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696742+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:41:27.696742+0000","last_peered":"2024-06-02T15:41:27.696742+0000","last_clean":"2024-06-02T15:41:27.696742+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:41:27.696742+0000","last_undegraded":"2024-06-02T15:41:27.696742+0000","last_fullsized":"2024-06-02T15:41:27.696742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699476+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:41:27.699476+0000","last_peered":"2024-06-02T15:41:27.699476+0000","last_clean":"2024-06-02T15:41:27.699476+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:41:27.699476+0000","last_undegraded":"2024-06-02T15:41:27.699476+0000","last_fullsized":"2024-06-02T15:41:27.699476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706081+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:41:27.706081+0000","last_peered":"2024-06-02T15:41:27.706081+0000","last_clean":"2024-06-02T15:41:27.706081+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:41:27.706081+0000","last_undegraded":"2024-06-02T15:41:27.706081+0000","last_fullsized":"2024-06-02T15:41:27.706081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702973+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:41:27.702973+0000","last_peered":"2024-06-02T15:41:27.702973+0000","last_clean":"2024-06-02T15:41:27.702973+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:41:27.702973+0000","last_undegraded":"2024-06-02T15:41:27.702973+0000","last_fullsized":"2024-06-02T15:41:27.702973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706900+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:41:27.706900+0000","last_peered":"2024-06-02T15:41:27.706900+0000","last_clean":"2024-06-02T15:41:27.706900+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:41:27.706900+0000","last_undegraded":"2024-06-02T15:41:27.706900+0000","last_fullsized":"2024-06-02T15:41:27.706900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702270+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:41:27.702270+0000","last_peered":"2024-06-02T15:41:27.702270+0000","last_clean":"2024-06-02T15:41:27.702270+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:41:27.702270+0000","last_undegraded":"2024-06-02T15:41:27.702270+0000","last_fullsized":"2024-06-02T15:41:27.702270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690254+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:41:27.690254+0000","last_peered":"2024-06-02T15:41:27.690254+0000","last_clean":"2024-06-02T15:41:27.690254+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:41:27.690254+0000","last_undegraded":"2024-06-02T15:41:27.690254+0000","last_fullsized":"2024-06-02T15:41:27.690254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:55.648352+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:55.648352+0000","last_peered":"2024-06-02T15:40:55.648352+0000","last_clean":"2024-06-02T15:40:55.648352+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:55.648352+0000","last_undegraded":"2024-06-02T15:40:55.648352+0000","last_fullsized":"2024-06-02T15:40:55.648352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849695+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:20.849695+0000","last_peered":"2024-06-02T15:40:20.849695+0000","last_clean":"2024-06-02T15:40:20.849695+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:20.849695+0000","last_undegraded":"2024-06-02T15:40:20.849695+0000","last_fullsized":"2024-06-02T15:40:20.849695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705339+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:41:27.705339+0000","last_peered":"2024-06-02T15:41:27.705339+0000","last_clean":"2024-06-02T15:41:27.705339+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:41:27.705339+0000","last_undegraded":"2024-06-02T15:41:27.705339+0000","last_fullsized":"2024-06-02T15:41:27.705339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.145879+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:54.145879+0000","last_peered":"2024-06-02T15:40:54.145879+0000","last_clean":"2024-06-02T15:40:54.145879+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:54.145879+0000","last_undegraded":"2024-06-02T15:40:54.145879+0000","last_fullsized":"2024-06-02T15:40:54.145879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:12.337384+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:12.337384+0000","last_peered":"2024-06-02T15:40:12.337384+0000","last_clean":"2024-06-02T15:40:12.337384+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:12.337384+0000","last_undegraded":"2024-06-02T15:40:12.337384+0000","last_fullsized":"2024-06-02T15:40:12.337384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708471+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:41:27.708471+0000","last_peered":"2024-06-02T15:41:27.708471+0000","last_clean":"2024-06-02T15:41:27.708471+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:41:27.708471+0000","last_undegraded":"2024-06-02T15:41:27.708471+0000","last_fullsized":"2024-06-02T15:41:27.708471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031438+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031438+0000","last_peered":"2024-06-02T15:40:00.031438+0000","last_clean":"2024-06-02T15:40:00.031438+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031438+0000","last_undegraded":"2024-06-02T15:40:00.031438+0000","last_fullsized":"2024-06-02T15:40:00.031438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850859+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:45.850859+0000","last_peered":"2024-06-02T15:40:45.850859+0000","last_clean":"2024-06-02T15:40:45.850859+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:45.850859+0000","last_undegraded":"2024-06-02T15:40:45.850859+0000","last_fullsized":"2024-06-02T15:40:45.850859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697673+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:41:27.697673+0000","last_peered":"2024-06-02T15:41:27.697673+0000","last_clean":"2024-06-02T15:41:27.697673+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:41:27.697673+0000","last_undegraded":"2024-06-02T15:41:27.697673+0000","last_fullsized":"2024-06-02T15:41:27.697673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031342+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031342+0000","last_peered":"2024-06-02T15:40:00.031342+0000","last_clean":"2024-06-02T15:40:00.031342+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031342+0000","last_undegraded":"2024-06-02T15:40:00.031342+0000","last_fullsized":"2024-06-02T15:40:00.031342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850848+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:45.850848+0000","last_peered":"2024-06-02T15:40:45.850848+0000","last_clean":"2024-06-02T15:40:45.850848+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:45.850848+0000","last_undegraded":"2024-06-02T15:40:45.850848+0000","last_fullsized":"2024-06-02T15:40:45.850848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696163+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:41:27.696163+0000","last_peered":"2024-06-02T15:41:27.696163+0000","last_clean":"2024-06-02T15:41:27.696163+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:41:27.696163+0000","last_undegraded":"2024-06-02T15:41:27.696163+0000","last_fullsized":"2024-06-02T15:41:27.696163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031260+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031260+0000","last_peered":"2024-06-02T15:40:00.031260+0000","last_clean":"2024-06-02T15:40:00.031260+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031260+0000","last_undegraded":"2024-06-02T15:40:00.031260+0000","last_fullsized":"2024-06-02T15:40:00.031260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":280,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.466055+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:43:17.466055+0000","last_peered":"2024-06-02T15:43:17.466055+0000","last_clean":"2024-06-02T15:43:17.466055+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:43:17.466055+0000","last_undegraded":"2024-06-02T15:43:17.466055+0000","last_fullsized":"2024-06-02T15:43:17.466055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":227,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697597+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:41:27.697597+0000","last_peered":"2024-06-02T15:41:27.697597+0000","last_clean":"2024-06-02T15:41:27.697597+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:41:27.697597+0000","last_undegraded":"2024-06-02T15:41:27.697597+0000","last_fullsized":"2024-06-02T15:41:27.697597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031151+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031151+0000","last_peered":"2024-06-02T15:40:00.031151+0000","last_clean":"2024-06-02T15:40:00.031151+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031151+0000","last_undegraded":"2024-06-02T15:40:00.031151+0000","last_fullsized":"2024-06-02T15:40:00.031151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:42.361651+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:42.361651+0000","last_peered":"2024-06-02T15:40:42.361651+0000","last_clean":"2024-06-02T15:40:42.361651+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:42.361651+0000","last_undegraded":"2024-06-02T15:40:42.361651+0000","last_fullsized":"2024-06-02T15:40:42.361651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711391+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:41:27.711391+0000","last_peered":"2024-06-02T15:41:27.711391+0000","last_clean":"2024-06-02T15:41:27.711391+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:41:27.711391+0000","last_undegraded":"2024-06-02T15:41:27.711391+0000","last_fullsized":"2024-06-02T15:41:27.711391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581825+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:34.581825+0000","last_peered":"2024-06-02T15:40:34.581825+0000","last_clean":"2024-06-02T15:40:34.581825+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:34.581825+0000","last_undegraded":"2024-06-02T15:40:34.581825+0000","last_fullsized":"2024-06-02T15:40:34.581825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463524+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:39.463524+0000","last_peered":"2024-06-02T15:40:39.463524+0000","last_clean":"2024-06-02T15:40:39.463524+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:39.463524+0000","last_undegraded":"2024-06-02T15:40:39.463524+0000","last_fullsized":"2024-06-02T15:40:39.463524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711431+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:41:27.711431+0000","last_peered":"2024-06-02T15:41:27.711431+0000","last_clean":"2024-06-02T15:41:27.711431+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:41:27.711431+0000","last_undegraded":"2024-06-02T15:41:27.711431+0000","last_fullsized":"2024-06-02T15:41:27.711431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:31.928337+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:31.928337+0000","last_peered":"2024-06-02T15:40:31.928337+0000","last_clean":"2024-06-02T15:40:31.928337+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:31.928337+0000","last_undegraded":"2024-06-02T15:40:31.928337+0000","last_fullsized":"2024-06-02T15:40:31.928337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929241+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:38.929241+0000","last_peered":"2024-06-02T15:40:38.929241+0000","last_clean":"2024-06-02T15:40:38.929241+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:38.929241+0000","last_undegraded":"2024-06-02T15:40:38.929241+0000","last_fullsized":"2024-06-02T15:40:38.929241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710382+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:41:27.710382+0000","last_peered":"2024-06-02T15:41:27.710382+0000","last_clean":"2024-06-02T15:41:27.710382+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:41:27.710382+0000","last_undegraded":"2024-06-02T15:41:27.710382+0000","last_fullsized":"2024-06-02T15:41:27.710382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.446626+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:50.446626+0000","last_peered":"2024-06-02T15:40:50.446626+0000","last_clean":"2024-06-02T15:40:50.446626+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:50.446626+0000","last_undegraded":"2024-06-02T15:40:50.446626+0000","last_fullsized":"2024-06-02T15:40:50.446626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":271,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:15.462318+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:43:15.462318+0000","last_peered":"2024-06-02T15:43:15.462318+0000","last_clean":"2024-06-02T15:43:15.462318+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:43:15.462318+0000","last_undegraded":"2024-06-02T15:43:15.462318+0000","last_fullsized":"2024-06-02T15:43:15.462318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":221,"num_read_kb":221,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701105+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:41:27.701105+0000","last_peered":"2024-06-02T15:41:27.701105+0000","last_clean":"2024-06-02T15:41:27.701105+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:41:27.701105+0000","last_undegraded":"2024-06-02T15:41:27.701105+0000","last_fullsized":"2024-06-02T15:41:27.701105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854789+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:50.854789+0000","last_peered":"2024-06-02T15:40:50.854789+0000","last_clean":"2024-06-02T15:40:50.854789+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:50.854789+0000","last_undegraded":"2024-06-02T15:40:50.854789+0000","last_fullsized":"2024-06-02T15:40:50.854789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:40.421522+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:40.421522+0000","last_peered":"2024-06-02T15:40:40.421522+0000","last_clean":"2024-06-02T15:40:40.421522+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:40.421522+0000","last_undegraded":"2024-06-02T15:40:40.421522+0000","last_fullsized":"2024-06-02T15:40:40.421522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706943+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:41:27.706943+0000","last_peered":"2024-06-02T15:41:27.706943+0000","last_clean":"2024-06-02T15:41:27.706943+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:41:27.706943+0000","last_undegraded":"2024-06-02T15:41:27.706943+0000","last_fullsized":"2024-06-02T15:41:27.706943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.145181+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:49.145181+0000","last_peered":"2024-06-02T15:40:49.145181+0000","last_clean":"2024-06-02T15:40:49.145181+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:49.145181+0000","last_undegraded":"2024-06-02T15:40:49.145181+0000","last_fullsized":"2024-06-02T15:40:49.145181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929293+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:38.929293+0000","last_peered":"2024-06-02T15:40:38.929293+0000","last_clean":"2024-06-02T15:40:38.929293+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:38.929293+0000","last_undegraded":"2024-06-02T15:40:38.929293+0000","last_fullsized":"2024-06-02T15:40:38.929293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698278+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:41:27.698278+0000","last_peered":"2024-06-02T15:41:27.698278+0000","last_clean":"2024-06-02T15:41:27.698278+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:41:27.698278+0000","last_undegraded":"2024-06-02T15:41:27.698278+0000","last_fullsized":"2024-06-02T15:41:27.698278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854862+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:50.854862+0000","last_peered":"2024-06-02T15:40:50.854862+0000","last_clean":"2024-06-02T15:40:50.854862+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:50.854862+0000","last_undegraded":"2024-06-02T15:40:50.854862+0000","last_fullsized":"2024-06-02T15:40:50.854862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":555,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:16.464411+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:43:16.464411+0000","last_peered":"2024-06-02T15:43:16.464411+0000","last_clean":"2024-06-02T15:43:16.464411+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:43:16.464411+0000","last_undegraded":"2024-06-02T15:43:16.464411+0000","last_fullsized":"2024-06-02T15:43:16.464411+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69988,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":453,"num_read_kb":290,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685871+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:41:27.685871+0000","last_peered":"2024-06-02T15:41:27.685871+0000","last_clean":"2024-06-02T15:41:27.685871+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:41:27.685871+0000","last_undegraded":"2024-06-02T15:41:27.685871+0000","last_fullsized":"2024-06-02T15:41:27.685871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.931198+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:53.931198+0000","last_peered":"2024-06-02T15:40:53.931198+0000","last_clean":"2024-06-02T15:40:53.931198+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:53.931198+0000","last_undegraded":"2024-06-02T15:40:53.931198+0000","last_fullsized":"2024-06-02T15:40:53.931198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646104+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:35.646104+0000","last_peered":"2024-06-02T15:40:35.646104+0000","last_clean":"2024-06-02T15:40:35.646104+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:35.646104+0000","last_undegraded":"2024-06-02T15:40:35.646104+0000","last_fullsized":"2024-06-02T15:40:35.646104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711870+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:41:27.711870+0000","last_peered":"2024-06-02T15:41:27.711870+0000","last_clean":"2024-06-02T15:41:27.711870+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:41:27.711870+0000","last_undegraded":"2024-06-02T15:41:27.711870+0000","last_fullsized":"2024-06-02T15:41:27.711870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.583429+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:49.583429+0000","last_peered":"2024-06-02T15:40:49.583429+0000","last_clean":"2024-06-02T15:40:49.583429+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:49.583429+0000","last_undegraded":"2024-06-02T15:40:49.583429+0000","last_fullsized":"2024-06-02T15:40:49.583429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:33.928487+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:33.928487+0000","last_peered":"2024-06-02T15:40:33.928487+0000","last_clean":"2024-06-02T15:40:33.928487+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:33.928487+0000","last_undegraded":"2024-06-02T15:40:33.928487+0000","last_fullsized":"2024-06-02T15:40:33.928487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696588+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:41:27.696588+0000","last_peered":"2024-06-02T15:41:27.696588+0000","last_clean":"2024-06-02T15:41:27.696588+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:41:27.696588+0000","last_undegraded":"2024-06-02T15:41:27.696588+0000","last_fullsized":"2024-06-02T15:41:27.696588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.911547+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:53.911547+0000","last_peered":"2024-06-02T15:40:53.911547+0000","last_clean":"2024-06-02T15:40:53.911547+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:53.911547+0000","last_undegraded":"2024-06-02T15:40:53.911547+0000","last_fullsized":"2024-06-02T15:40:53.911547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":278,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615898+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:43:17.615898+0000","last_peered":"2024-06-02T15:43:17.615898+0000","last_clean":"2024-06-02T15:43:17.615898+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:43:17.615898+0000","last_undegraded":"2024-06-02T15:43:17.615898+0000","last_fullsized":"2024-06-02T15:43:17.615898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687560+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:41:27.687560+0000","last_peered":"2024-06-02T15:41:27.687560+0000","last_clean":"2024-06-02T15:41:27.687560+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:41:27.687560+0000","last_undegraded":"2024-06-02T15:41:27.687560+0000","last_fullsized":"2024-06-02T15:41:27.687560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.466042+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:54.466042+0000","last_peered":"2024-06-02T15:40:54.466042+0000","last_clean":"2024-06-02T15:40:54.466042+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:54.466042+0000","last_undegraded":"2024-06-02T15:40:54.466042+0000","last_fullsized":"2024-06-02T15:40:54.466042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849838+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:35.849838+0000","last_peered":"2024-06-02T15:40:35.849838+0000","last_clean":"2024-06-02T15:40:35.849838+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:35.849838+0000","last_undegraded":"2024-06-02T15:40:35.849838+0000","last_fullsized":"2024-06-02T15:40:35.849838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694458+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:41:27.694458+0000","last_peered":"2024-06-02T15:41:27.694458+0000","last_clean":"2024-06-02T15:41:27.694458+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:41:27.694458+0000","last_undegraded":"2024-06-02T15:41:27.694458+0000","last_fullsized":"2024-06-02T15:41:27.694458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929257+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:38.929257+0000","last_peered":"2024-06-02T15:40:38.929257+0000","last_clean":"2024-06-02T15:40:38.929257+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:38.929257+0000","last_undegraded":"2024-06-02T15:40:38.929257+0000","last_fullsized":"2024-06-02T15:40:38.929257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849304+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:30.849304+0000","last_peered":"2024-06-02T15:40:30.849304+0000","last_clean":"2024-06-02T15:40:30.849304+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:30.849304+0000","last_undegraded":"2024-06-02T15:40:30.849304+0000","last_fullsized":"2024-06-02T15:40:30.849304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695493+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:41:27.695493+0000","last_peered":"2024-06-02T15:41:27.695493+0000","last_clean":"2024-06-02T15:41:27.695493+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:41:27.695493+0000","last_undegraded":"2024-06-02T15:41:27.695493+0000","last_fullsized":"2024-06-02T15:41:27.695493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849864+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:35.849864+0000","last_peered":"2024-06-02T15:40:35.849864+0000","last_clean":"2024-06-02T15:40:35.849864+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:35.849864+0000","last_undegraded":"2024-06-02T15:40:35.849864+0000","last_fullsized":"2024-06-02T15:40:35.849864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849635+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:20.849635+0000","last_peered":"2024-06-02T15:40:20.849635+0000","last_clean":"2024-06-02T15:40:20.849635+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:20.849635+0000","last_undegraded":"2024-06-02T15:40:20.849635+0000","last_fullsized":"2024-06-02T15:40:20.849635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693299+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:41:27.693299+0000","last_peered":"2024-06-02T15:41:27.693299+0000","last_clean":"2024-06-02T15:41:27.693299+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:41:27.693299+0000","last_undegraded":"2024-06-02T15:41:27.693299+0000","last_fullsized":"2024-06-02T15:41:27.693299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.462890+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:34.462890+0000","last_peered":"2024-06-02T15:40:34.462890+0000","last_clean":"2024-06-02T15:40:34.462890+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:34.462890+0000","last_undegraded":"2024-06-02T15:40:34.462890+0000","last_fullsized":"2024-06-02T15:40:34.462890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.419076+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:20.419076+0000","last_peered":"2024-06-02T15:40:20.419076+0000","last_clean":"2024-06-02T15:40:20.419076+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:20.419076+0000","last_undegraded":"2024-06-02T15:40:20.419076+0000","last_fullsized":"2024-06-02T15:40:20.419076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.676935+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:41:27.676935+0000","last_peered":"2024-06-02T15:41:27.676935+0000","last_clean":"2024-06-02T15:41:27.676935+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:41:27.676935+0000","last_undegraded":"2024-06-02T15:41:27.676935+0000","last_fullsized":"2024-06-02T15:41:27.676935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463609+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:39.463609+0000","last_peered":"2024-06-02T15:40:39.463609+0000","last_clean":"2024-06-02T15:40:39.463609+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:39.463609+0000","last_undegraded":"2024-06-02T15:40:39.463609+0000","last_fullsized":"2024-06-02T15:40:39.463609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690130+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:41:27.690130+0000","last_peered":"2024-06-02T15:41:27.690130+0000","last_clean":"2024-06-02T15:41:27.690130+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:41:27.690130+0000","last_undegraded":"2024-06-02T15:41:27.690130+0000","last_fullsized":"2024-06-02T15:41:27.690130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":274,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615195+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:43:17.615195+0000","last_peered":"2024-06-02T15:43:17.615195+0000","last_clean":"2024-06-02T15:43:17.615195+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:43:17.615195+0000","last_undegraded":"2024-06-02T15:43:17.615195+0000","last_fullsized":"2024-06-02T15:43:17.615195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":223,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.420757+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:35.420757+0000","last_peered":"2024-06-02T15:40:35.420757+0000","last_clean":"2024-06-02T15:40:35.420757+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:35.420757+0000","last_undegraded":"2024-06-02T15:40:35.420757+0000","last_fullsized":"2024-06-02T15:40:35.420757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708881+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:41:27.708881+0000","last_peered":"2024-06-02T15:41:27.708881+0000","last_clean":"2024-06-02T15:41:27.708881+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:41:27.708881+0000","last_undegraded":"2024-06-02T15:41:27.708881+0000","last_fullsized":"2024-06-02T15:41:27.708881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:19.142002+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:19.142002+0000","last_peered":"2024-06-02T15:40:19.142002+0000","last_clean":"2024-06-02T15:40:19.142002+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:19.142002+0000","last_undegraded":"2024-06-02T15:40:19.142002+0000","last_fullsized":"2024-06-02T15:40:19.142002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:10.424476+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:41:10.424476+0000","last_peered":"2024-06-02T15:41:10.424476+0000","last_clean":"2024-06-02T15:41:10.424476+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:41:10.424476+0000","last_undegraded":"2024-06-02T15:41:10.424476+0000","last_fullsized":"2024-06-02T15:41:10.424476+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929309+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:38.929309+0000","last_peered":"2024-06-02T15:40:38.929309+0000","last_clean":"2024-06-02T15:40:38.929309+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:38.929309+0000","last_undegraded":"2024-06-02T15:40:38.929309+0000","last_fullsized":"2024-06-02T15:40:38.929309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685503+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:41:27.685503+0000","last_peered":"2024-06-02T15:41:27.685503+0000","last_clean":"2024-06-02T15:41:27.685503+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:41:27.685503+0000","last_undegraded":"2024-06-02T15:41:27.685503+0000","last_fullsized":"2024-06-02T15:41:27.685503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:15.418582+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:15.418582+0000","last_peered":"2024-06-02T15:40:15.418582+0000","last_clean":"2024-06-02T15:40:15.418582+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:15.418582+0000","last_undegraded":"2024-06-02T15:40:15.418582+0000","last_fullsized":"2024-06-02T15:40:15.418582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581781+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:34.581781+0000","last_peered":"2024-06-02T15:40:34.581781+0000","last_clean":"2024-06-02T15:40:34.581781+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:34.581781+0000","last_undegraded":"2024-06-02T15:40:34.581781+0000","last_fullsized":"2024-06-02T15:40:34.581781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.848207+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:20.848207+0000","last_peered":"2024-06-02T15:40:20.848207+0000","last_clean":"2024-06-02T15:40:20.848207+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:20.848207+0000","last_undegraded":"2024-06-02T15:40:20.848207+0000","last_fullsized":"2024-06-02T15:40:20.848207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693951+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:41:27.693951+0000","last_peered":"2024-06-02T15:41:27.693951+0000","last_clean":"2024-06-02T15:41:27.693951+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:41:27.693951+0000","last_undegraded":"2024-06-02T15:41:27.693951+0000","last_fullsized":"2024-06-02T15:41:27.693951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:32.340299+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:32.340299+0000","last_peered":"2024-06-02T15:40:32.340299+0000","last_clean":"2024-06-02T15:40:32.340299+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:32.340299+0000","last_undegraded":"2024-06-02T15:40:32.340299+0000","last_fullsized":"2024-06-02T15:40:32.340299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:42:35.870244+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:42:35.870244+0000","last_peered":"2024-06-02T15:42:35.870244+0000","last_clean":"2024-06-02T15:42:35.870244+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:42:35.870244+0000","last_undegraded":"2024-06-02T15:42:35.870244+0000","last_fullsized":"2024-06-02T15:42:35.870244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.709823+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:41:27.709823+0000","last_peered":"2024-06-02T15:41:27.709823+0000","last_clean":"2024-06-02T15:41:27.709823+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:41:27.709823+0000","last_undegraded":"2024-06-02T15:41:27.709823+0000","last_fullsized":"2024-06-02T15:41:27.709823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646076+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:35.646076+0000","last_peered":"2024-06-02T15:40:35.646076+0000","last_clean":"2024-06-02T15:40:35.646076+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:35.646076+0000","last_undegraded":"2024-06-02T15:40:35.646076+0000","last_fullsized":"2024-06-02T15:40:35.646076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:24.142446+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:24.142446+0000","last_peered":"2024-06-02T15:40:24.142446+0000","last_clean":"2024-06-02T15:40:24.142446+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:24.142446+0000","last_undegraded":"2024-06-02T15:40:24.142446+0000","last_fullsized":"2024-06-02T15:40:24.142446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711979+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:41:27.711979+0000","last_peered":"2024-06-02T15:41:27.711979+0000","last_clean":"2024-06-02T15:41:27.711979+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:41:27.711979+0000","last_undegraded":"2024-06-02T15:41:27.711979+0000","last_fullsized":"2024-06-02T15:41:27.711979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144738+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:44.144738+0000","last_peered":"2024-06-02T15:40:44.144738+0000","last_clean":"2024-06-02T15:40:44.144738+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:44.144738+0000","last_undegraded":"2024-06-02T15:40:44.144738+0000","last_fullsized":"2024-06-02T15:40:44.144738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:26.381402+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:26.381402+0000","last_peered":"2024-06-02T15:40:26.381402+0000","last_clean":"2024-06-02T15:40:26.381402+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:26.381402+0000","last_undegraded":"2024-06-02T15:40:26.381402+0000","last_fullsized":"2024-06-02T15:40:26.381402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.680443+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:41:27.680443+0000","last_peered":"2024-06-02T15:41:27.680443+0000","last_clean":"2024-06-02T15:41:27.680443+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:41:27.680443+0000","last_undegraded":"2024-06-02T15:41:27.680443+0000","last_fullsized":"2024-06-02T15:41:27.680443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.853966+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:45.853966+0000","last_peered":"2024-06-02T15:40:45.853966+0000","last_clean":"2024-06-02T15:40:45.853966+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:45.853966+0000","last_undegraded":"2024-06-02T15:40:45.853966+0000","last_fullsized":"2024-06-02T15:40:45.853966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:22.338983+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:22.338983+0000","last_peered":"2024-06-02T15:40:22.338983+0000","last_clean":"2024-06-02T15:40:22.338983+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:22.338983+0000","last_undegraded":"2024-06-02T15:40:22.338983+0000","last_fullsized":"2024-06-02T15:40:22.338983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703948+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:41:27.703948+0000","last_peered":"2024-06-02T15:41:27.703948+0000","last_clean":"2024-06-02T15:41:27.703948+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:41:27.703948+0000","last_undegraded":"2024-06-02T15:41:27.703948+0000","last_fullsized":"2024-06-02T15:41:27.703948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144717+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:44.144717+0000","last_peered":"2024-06-02T15:40:44.144717+0000","last_clean":"2024-06-02T15:40:44.144717+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:44.144717+0000","last_undegraded":"2024-06-02T15:40:44.144717+0000","last_fullsized":"2024-06-02T15:40:44.144717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:09.585558+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:41:09.585558+0000","last_peered":"2024-06-02T15:41:09.585558+0000","last_clean":"2024-06-02T15:41:09.585558+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:41:09.585558+0000","last_undegraded":"2024-06-02T15:41:09.585558+0000","last_fullsized":"2024-06-02T15:41:09.585558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711497+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:41:27.711497+0000","last_peered":"2024-06-02T15:41:27.711497+0000","last_clean":"2024-06-02T15:41:27.711497+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:41:27.711497+0000","last_undegraded":"2024-06-02T15:41:27.711497+0000","last_fullsized":"2024-06-02T15:41:27.711497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144729+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:44.144729+0000","last_peered":"2024-06-02T15:40:44.144729+0000","last_clean":"2024-06-02T15:40:44.144729+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:44.144729+0000","last_undegraded":"2024-06-02T15:40:44.144729+0000","last_fullsized":"2024-06-02T15:40:44.144729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849271+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:30.849271+0000","last_peered":"2024-06-02T15:40:30.849271+0000","last_clean":"2024-06-02T15:40:30.849271+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:30.849271+0000","last_undegraded":"2024-06-02T15:40:30.849271+0000","last_fullsized":"2024-06-02T15:40:30.849271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692818+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:41:27.692818+0000","last_peered":"2024-06-02T15:41:27.692818+0000","last_clean":"2024-06-02T15:41:27.692818+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:41:27.692818+0000","last_undegraded":"2024-06-02T15:41:27.692818+0000","last_fullsized":"2024-06-02T15:41:27.692818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:41.383274+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:41.383274+0000","last_peered":"2024-06-02T15:40:41.383274+0000","last_clean":"2024-06-02T15:40:41.383274+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:41.383274+0000","last_undegraded":"2024-06-02T15:40:41.383274+0000","last_fullsized":"2024-06-02T15:40:41.383274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.645527+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:30.645527+0000","last_peered":"2024-06-02T15:40:30.645527+0000","last_clean":"2024-06-02T15:40:30.645527+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:30.645527+0000","last_undegraded":"2024-06-02T15:40:30.645527+0000","last_fullsized":"2024-06-02T15:40:30.645527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694463+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:41:27.694463+0000","last_peered":"2024-06-02T15:41:27.694463+0000","last_clean":"2024-06-02T15:41:27.694463+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:41:27.694463+0000","last_undegraded":"2024-06-02T15:41:27.694463+0000","last_fullsized":"2024-06-02T15:41:27.694463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:46.930876+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:46.930876+0000","last_peered":"2024-06-02T15:40:46.930876+0000","last_clean":"2024-06-02T15:40:46.930876+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:46.930876+0000","last_undegraded":"2024-06-02T15:40:46.930876+0000","last_fullsized":"2024-06-02T15:40:46.930876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:28.928032+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:28.928032+0000","last_peered":"2024-06-02T15:40:28.928032+0000","last_clean":"2024-06-02T15:40:28.928032+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:28.928032+0000","last_undegraded":"2024-06-02T15:40:28.928032+0000","last_fullsized":"2024-06-02T15:40:28.928032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705291+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:41:27.705291+0000","last_peered":"2024-06-02T15:41:27.705291+0000","last_clean":"2024-06-02T15:41:27.705291+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:41:27.705291+0000","last_undegraded":"2024-06-02T15:41:27.705291+0000","last_fullsized":"2024-06-02T15:41:27.705291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929210+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:38.929210+0000","last_peered":"2024-06-02T15:40:38.929210+0000","last_clean":"2024-06-02T15:40:38.929210+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:38.929210+0000","last_undegraded":"2024-06-02T15:40:38.929210+0000","last_fullsized":"2024-06-02T15:40:38.929210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:29.462003+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:29.462003+0000","last_peered":"2024-06-02T15:40:29.462003+0000","last_clean":"2024-06-02T15:40:29.462003+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:29.462003+0000","last_undegraded":"2024-06-02T15:40:29.462003+0000","last_fullsized":"2024-06-02T15:40:29.462003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685479+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:41:27.685479+0000","last_peered":"2024-06-02T15:41:27.685479+0000","last_clean":"2024-06-02T15:41:27.685479+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:41:27.685479+0000","last_undegraded":"2024-06-02T15:41:27.685479+0000","last_fullsized":"2024-06-02T15:41:27.685479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687491+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:41:27.687491+0000","last_peered":"2024-06-02T15:41:27.687491+0000","last_clean":"2024-06-02T15:41:27.687491+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:41:27.687491+0000","last_undegraded":"2024-06-02T15:41:27.687491+0000","last_fullsized":"2024-06-02T15:41:27.687491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686399+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:41:27.686399+0000","last_peered":"2024-06-02T15:41:27.686399+0000","last_clean":"2024-06-02T15:41:27.686399+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:41:27.686399+0000","last_undegraded":"2024-06-02T15:41:27.686399+0000","last_fullsized":"2024-06-02T15:41:27.686399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701372+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:41:27.701372+0000","last_peered":"2024-06-02T15:41:27.701372+0000","last_clean":"2024-06-02T15:41:27.701372+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:41:27.701372+0000","last_undegraded":"2024-06-02T15:41:27.701372+0000","last_fullsized":"2024-06-02T15:41:27.701372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701977+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:41:27.701977+0000","last_peered":"2024-06-02T15:41:27.701977+0000","last_clean":"2024-06-02T15:41:27.701977+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:41:27.701977+0000","last_undegraded":"2024-06-02T15:41:27.701977+0000","last_fullsized":"2024-06-02T15:41:27.701977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704468+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:41:27.704468+0000","last_peered":"2024-06-02T15:41:27.704468+0000","last_clean":"2024-06-02T15:41:27.704468+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:41:27.704468+0000","last_undegraded":"2024-06-02T15:41:27.704468+0000","last_fullsized":"2024-06-02T15:41:27.704468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686919+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:41:27.686919+0000","last_peered":"2024-06-02T15:41:27.686919+0000","last_clean":"2024-06-02T15:41:27.686919+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:41:27.686919+0000","last_undegraded":"2024-06-02T15:41:27.686919+0000","last_fullsized":"2024-06-02T15:41:27.686919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.691250+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:41:27.691250+0000","last_peered":"2024-06-02T15:41:27.691250+0000","last_clean":"2024-06-02T15:41:27.691250+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:41:27.691250+0000","last_undegraded":"2024-06-02T15:41:27.691250+0000","last_fullsized":"2024-06-02T15:41:27.691250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702170+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:41:27.702170+0000","last_peered":"2024-06-02T15:41:27.702170+0000","last_clean":"2024-06-02T15:41:27.702170+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:41:27.702170+0000","last_undegraded":"2024-06-02T15:41:27.702170+0000","last_fullsized":"2024-06-02T15:41:27.702170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711806+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:41:27.711806+0000","last_peered":"2024-06-02T15:41:27.711806+0000","last_clean":"2024-06-02T15:41:27.711806+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:41:27.711806+0000","last_undegraded":"2024-06-02T15:41:27.711806+0000","last_fullsized":"2024-06-02T15:41:27.711806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706949+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:41:27.706949+0000","last_peered":"2024-06-02T15:41:27.706949+0000","last_clean":"2024-06-02T15:41:27.706949+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:41:27.706949+0000","last_undegraded":"2024-06-02T15:41:27.706949+0000","last_fullsized":"2024-06-02T15:41:27.706949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711917+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:41:27.711917+0000","last_peered":"2024-06-02T15:41:27.711917+0000","last_clean":"2024-06-02T15:41:27.711917+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:41:27.711917+0000","last_undegraded":"2024-06-02T15:41:27.711917+0000","last_fullsized":"2024-06-02T15:41:27.711917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703925+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:41:27.703925+0000","last_peered":"2024-06-02T15:41:27.703925+0000","last_clean":"2024-06-02T15:41:27.703925+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:41:27.703925+0000","last_undegraded":"2024-06-02T15:41:27.703925+0000","last_fullsized":"2024-06-02T15:41:27.703925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698726+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:41:27.698726+0000","last_peered":"2024-06-02T15:41:27.698726+0000","last_clean":"2024-06-02T15:41:27.698726+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:41:27.698726+0000","last_undegraded":"2024-06-02T15:41:27.698726+0000","last_fullsized":"2024-06-02T15:41:27.698726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74662,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1365,"num_read_kb":756,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874323,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1728512,"data_stored":1175948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037850,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430309,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1114316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593835,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":583470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":515364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149819,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953591,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":588742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790065,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1105858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:43:23.549 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-02T15:43:23.550 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T15:43:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:23 smithi120 ceph-mon[36451]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-02T15:43:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:23 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:43:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:23 smithi135 ceph-mon[36839]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 5 op/s 2024-06-02T15:43:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:23 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:43:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 -- 172.21.15.90:0/1666501580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1028f0 msgr2=0x7f5d3c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 --2- 172.21.15.90:0/1666501580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c102d10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009ca0 tx=0x7f5d2c02a8c0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 -- 172.21.15.90:0/1666501580 shutdown_connections 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 --2- 172.21.15.90:0/1666501580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 --2- 172.21.15.90:0/1666501580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 --2- 172.21.15.90:0/1666501580 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c102d10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 -- 172.21.15.90:0/1666501580 >> 172.21.15.90:0/1666501580 conn(0x7f5d3c0fde60 msgr2=0x7f5d3c1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 -- 172.21.15.90:0/1666501580 shutdown_connections 2024-06-02T15:43:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.790+0000 7f5d41800640 1 -- 172.21.15.90:0/1666501580 wait complete. 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.791+0000 7f5d41800640 1 Processor -- start 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 -- start start 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e470 con 0x7f5d3c103ae0 2024-06-02T15:43:23.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e2f0 con 0x7f5d3c1044a0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d41800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e5f0 con 0x7f5d3c1028f0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d40e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34704/0 (socket says 172.21.15.90:34704) 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d40e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47058/0 (socket says 172.21.15.90:47058) 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d33400640 1 -- 172.21.15.90:0/1012673307 learned_addr learned my addr 172.21.15.90:0/1012673307 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.792+0000 7f5d3be00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 -- 172.21.15.90:0/1012673307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1028f0 msgr2=0x7f5d3c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 -- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 msgr2=0x7f5d3c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 -- 172.21.15.90:0/1012673307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a2950 con 0x7f5d3c1044a0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d40e00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e970 tx=0x7f5d3400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400ccc0 con 0x7f5d3c1044a0 2024-06-02T15:43:23.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34005020 con 0x7f5d3c1044a0 2024-06-02T15:43:23.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34004a00 con 0x7f5d3c1044a0 2024-06-02T15:43:23.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a2c40 con 0x7f5d3c1044a0 2024-06-02T15:43:23.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.793+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a3110 con 0x7f5d3c1044a0 2024-06-02T15:43:23.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.794+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c1a3340 con 0x7f5d3c1044a0 2024-06-02T15:43:23.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.797+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5d34004ba0 con 0x7f5d3c1044a0 2024-06-02T15:43:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.799+0000 7f5d39600640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 0x7f5d200b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.799+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5d340e2560 con 0x7f5d3c1044a0 2024-06-02T15:43:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.799+0000 7f5d3be00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 0x7f5d200b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:23.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.800+0000 7f5d3be00640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 0x7f5d200b4d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f80 tx=0x7f5d2c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:23.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:23.803+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d340a9e70 con 0x7f5d3c1044a0 2024-06-02T15:43:24.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.011+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 --> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d3c1a3510 con 0x7f5d200b2870 2024-06-02T15:43:24.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.039+0000 7f5d39600640 1 -- 172.21.15.90:0/1012673307 <== mgr.14152 v2:172.21.15.90:6800/436264481 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340486 (secure 0 0 0) 0x7f5d3c1a3510 con 0x7f5d200b2870 2024-06-02T15:43:24.042 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:24.044 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 msgr2=0x7f5d200b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 0x7f5d200b4d30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c000f80 tx=0x7f5d2c005f50 comp rx=0 tx=0).stop 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 msgr2=0x7f5d3c1a22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e970 tx=0x7f5d3400ee30 comp rx=0 tx=0).stop 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 shutdown_connections 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5d200b2870 0x7f5d200b4d30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5d3c1044a0 0x7f5d3c1a22b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d3c103ae0 0x7f5d3c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:24.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 --2- 172.21.15.90:0/1012673307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d3c1028f0 0x7f5d3c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:24.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 >> 172.21.15.90:0/1012673307 conn(0x7f5d3c0fde60 msgr2=0x7f5d3c074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:24.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 shutdown_connections 2024-06-02T15:43:24.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:24.047+0000 7f5d41800640 1 -- 172.21.15.90:0/1012673307 wait complete. 2024-06-02T15:43:24.064 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-06-02T15:43:23.826729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665264,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1457,"num_read_kb":836,"num_write":310,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":2954151,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":752,"ondisk_log_size":752,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375836,"kb_used_data":14204,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565988,"statfs":{"total":1151940427776,"available":1151555571712,"internally_reserved":0,"allocated":14544896,"data_stored":8189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19139,"internal_metadata":370193725},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5679,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006703"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706882+0000","last_change":"2024-06-02T15:38:16.472986+0000","last_active":"2024-06-02T15:41:27.706882+0000","last_peered":"2024-06-02T15:41:27.706882+0000","last_clean":"2024-06-02T15:41:27.706882+0000","last_became_active":"2024-06-02T15:38:16.472100+0000","last_became_peered":"2024-06-02T15:38:16.472100+0000","last_unstale":"2024-06-02T15:41:27.706882+0000","last_undegraded":"2024-06-02T15:41:27.706882+0000","last_fullsized":"2024-06-02T15:41:27.706882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:17:26.484002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706280+0000","last_change":"2024-06-02T15:38:16.471679+0000","last_active":"2024-06-02T15:41:27.706280+0000","last_peered":"2024-06-02T15:41:27.706280+0000","last_clean":"2024-06-02T15:41:27.706280+0000","last_became_active":"2024-06-02T15:38:16.468109+0000","last_became_peered":"2024-06-02T15:38:16.468109+0000","last_unstale":"2024-06-02T15:41:27.706280+0000","last_undegraded":"2024-06-02T15:41:27.706280+0000","last_fullsized":"2024-06-02T15:41:27.706280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:57:23.615412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706538+0000","last_change":"2024-06-02T15:38:16.467113+0000","last_active":"2024-06-02T15:41:27.706538+0000","last_peered":"2024-06-02T15:41:27.706538+0000","last_clean":"2024-06-02T15:41:27.706538+0000","last_became_active":"2024-06-02T15:38:16.466752+0000","last_became_peered":"2024-06-02T15:38:16.466752+0000","last_unstale":"2024-06-02T15:41:27.706538+0000","last_undegraded":"2024-06-02T15:41:27.706538+0000","last_fullsized":"2024-06-02T15:41:27.706538+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:43:29.311419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692190+0000","last_change":"2024-06-02T15:38:16.464841+0000","last_active":"2024-06-02T15:41:27.692190+0000","last_peered":"2024-06-02T15:41:27.692190+0000","last_clean":"2024-06-02T15:41:27.692190+0000","last_became_active":"2024-06-02T15:38:16.463496+0000","last_became_peered":"2024-06-02T15:38:16.463496+0000","last_unstale":"2024-06-02T15:41:27.692190+0000","last_undegraded":"2024-06-02T15:41:27.692190+0000","last_fullsized":"2024-06-02T15:41:27.692190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:40:58.627155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699200+0000","last_change":"2024-06-02T15:38:16.481514+0000","last_active":"2024-06-02T15:41:27.699200+0000","last_peered":"2024-06-02T15:41:27.699200+0000","last_clean":"2024-06-02T15:41:27.699200+0000","last_became_active":"2024-06-02T15:38:16.481133+0000","last_became_peered":"2024-06-02T15:38:16.481133+0000","last_unstale":"2024-06-02T15:41:27.699200+0000","last_undegraded":"2024-06-02T15:41:27.699200+0000","last_fullsized":"2024-06-02T15:41:27.699200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:43.668615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697522+0000","last_change":"2024-06-02T15:38:16.480244+0000","last_active":"2024-06-02T15:41:27.697522+0000","last_peered":"2024-06-02T15:41:27.697522+0000","last_clean":"2024-06-02T15:41:27.697522+0000","last_became_active":"2024-06-02T15:38:16.479861+0000","last_became_peered":"2024-06-02T15:38:16.479861+0000","last_unstale":"2024-06-02T15:41:27.697522+0000","last_undegraded":"2024-06-02T15:41:27.697522+0000","last_fullsized":"2024-06-02T15:41:27.697522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:50.308218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.700721+0000","last_change":"2024-06-02T15:38:16.469292+0000","last_active":"2024-06-02T15:41:27.700721+0000","last_peered":"2024-06-02T15:41:27.700721+0000","last_clean":"2024-06-02T15:41:27.700721+0000","last_became_active":"2024-06-02T15:38:16.468952+0000","last_became_peered":"2024-06-02T15:38:16.468952+0000","last_unstale":"2024-06-02T15:41:27.700721+0000","last_undegraded":"2024-06-02T15:41:27.700721+0000","last_fullsized":"2024-06-02T15:41:27.700721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:23:06.408508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711556+0000","last_change":"2024-06-02T15:38:16.479304+0000","last_active":"2024-06-02T15:41:27.711556+0000","last_peered":"2024-06-02T15:41:27.711556+0000","last_clean":"2024-06-02T15:41:27.711556+0000","last_became_active":"2024-06-02T15:38:16.476514+0000","last_became_peered":"2024-06-02T15:38:16.476514+0000","last_unstale":"2024-06-02T15:41:27.711556+0000","last_undegraded":"2024-06-02T15:41:27.711556+0000","last_fullsized":"2024-06-02T15:41:27.711556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:27:56.833368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692972+0000","last_change":"2024-06-02T15:38:59.184373+0000","last_active":"2024-06-02T15:41:27.692972+0000","last_peered":"2024-06-02T15:41:27.692972+0000","last_clean":"2024-06-02T15:41:27.692972+0000","last_became_active":"2024-06-02T15:38:59.183931+0000","last_became_peered":"2024-06-02T15:38:59.183931+0000","last_unstale":"2024-06-02T15:41:27.692972+0000","last_undegraded":"2024-06-02T15:41:27.692972+0000","last_fullsized":"2024-06-02T15:41:27.692972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:56:41.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698817+0000","last_change":"2024-06-02T15:38:59.184249+0000","last_active":"2024-06-02T15:41:27.698817+0000","last_peered":"2024-06-02T15:41:27.698817+0000","last_clean":"2024-06-02T15:41:27.698817+0000","last_became_active":"2024-06-02T15:38:59.183667+0000","last_became_peered":"2024-06-02T15:38:59.183667+0000","last_unstale":"2024-06-02T15:41:27.698817+0000","last_undegraded":"2024-06-02T15:41:27.698817+0000","last_fullsized":"2024-06-02T15:41:27.698817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:12:28.821216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695499+0000","last_change":"2024-06-02T15:38:16.463685+0000","last_active":"2024-06-02T15:41:27.695499+0000","last_peered":"2024-06-02T15:41:27.695499+0000","last_clean":"2024-06-02T15:41:27.695499+0000","last_became_active":"2024-06-02T15:38:16.463340+0000","last_became_peered":"2024-06-02T15:38:16.463340+0000","last_unstale":"2024-06-02T15:41:27.695499+0000","last_undegraded":"2024-06-02T15:41:27.695499+0000","last_fullsized":"2024-06-02T15:41:27.695499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:23.545040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690188+0000","last_change":"2024-06-02T15:38:16.457120+0000","last_active":"2024-06-02T15:41:27.690188+0000","last_peered":"2024-06-02T15:41:27.690188+0000","last_clean":"2024-06-02T15:41:27.690188+0000","last_became_active":"2024-06-02T15:38:16.456192+0000","last_became_peered":"2024-06-02T15:38:16.456192+0000","last_unstale":"2024-06-02T15:41:27.690188+0000","last_undegraded":"2024-06-02T15:41:27.690188+0000","last_fullsized":"2024-06-02T15:41:27.690188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:04:56.809650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696742+0000","last_change":"2024-06-02T15:38:16.481284+0000","last_active":"2024-06-02T15:41:27.696742+0000","last_peered":"2024-06-02T15:41:27.696742+0000","last_clean":"2024-06-02T15:41:27.696742+0000","last_became_active":"2024-06-02T15:38:16.480829+0000","last_became_peered":"2024-06-02T15:38:16.480829+0000","last_unstale":"2024-06-02T15:41:27.696742+0000","last_undegraded":"2024-06-02T15:41:27.696742+0000","last_fullsized":"2024-06-02T15:41:27.696742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:10:29.407102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.699476+0000","last_change":"2024-06-02T15:38:16.468495+0000","last_active":"2024-06-02T15:41:27.699476+0000","last_peered":"2024-06-02T15:41:27.699476+0000","last_clean":"2024-06-02T15:41:27.699476+0000","last_became_active":"2024-06-02T15:38:16.463978+0000","last_became_peered":"2024-06-02T15:38:16.463978+0000","last_unstale":"2024-06-02T15:41:27.699476+0000","last_undegraded":"2024-06-02T15:41:27.699476+0000","last_fullsized":"2024-06-02T15:41:27.699476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:20:37.504154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706081+0000","last_change":"2024-06-02T15:38:16.470571+0000","last_active":"2024-06-02T15:41:27.706081+0000","last_peered":"2024-06-02T15:41:27.706081+0000","last_clean":"2024-06-02T15:41:27.706081+0000","last_became_active":"2024-06-02T15:38:16.467245+0000","last_became_peered":"2024-06-02T15:38:16.467245+0000","last_unstale":"2024-06-02T15:41:27.706081+0000","last_undegraded":"2024-06-02T15:41:27.706081+0000","last_fullsized":"2024-06-02T15:41:27.706081+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:34:23.614620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702973+0000","last_change":"2024-06-02T15:38:16.477387+0000","last_active":"2024-06-02T15:41:27.702973+0000","last_peered":"2024-06-02T15:41:27.702973+0000","last_clean":"2024-06-02T15:41:27.702973+0000","last_became_active":"2024-06-02T15:38:16.476994+0000","last_became_peered":"2024-06-02T15:38:16.476994+0000","last_unstale":"2024-06-02T15:41:27.702973+0000","last_undegraded":"2024-06-02T15:41:27.702973+0000","last_fullsized":"2024-06-02T15:41:27.702973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:24:44.394358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706900+0000","last_change":"2024-06-02T15:38:16.480674+0000","last_active":"2024-06-02T15:41:27.706900+0000","last_peered":"2024-06-02T15:41:27.706900+0000","last_clean":"2024-06-02T15:41:27.706900+0000","last_became_active":"2024-06-02T15:38:16.480288+0000","last_became_peered":"2024-06-02T15:38:16.480288+0000","last_unstale":"2024-06-02T15:41:27.706900+0000","last_undegraded":"2024-06-02T15:41:27.706900+0000","last_fullsized":"2024-06-02T15:41:27.706900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:55:37.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702270+0000","last_change":"2024-06-02T15:38:16.468797+0000","last_active":"2024-06-02T15:41:27.702270+0000","last_peered":"2024-06-02T15:41:27.702270+0000","last_clean":"2024-06-02T15:41:27.702270+0000","last_became_active":"2024-06-02T15:38:16.468334+0000","last_became_peered":"2024-06-02T15:38:16.468334+0000","last_unstale":"2024-06-02T15:41:27.702270+0000","last_undegraded":"2024-06-02T15:41:27.702270+0000","last_fullsized":"2024-06-02T15:41:27.702270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:05:42.953095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690254+0000","last_change":"2024-06-02T15:39:00.131812+0000","last_active":"2024-06-02T15:41:27.690254+0000","last_peered":"2024-06-02T15:41:27.690254+0000","last_clean":"2024-06-02T15:41:27.690254+0000","last_became_active":"2024-06-02T15:39:00.131516+0000","last_became_peered":"2024-06-02T15:39:00.131516+0000","last_unstale":"2024-06-02T15:41:27.690254+0000","last_undegraded":"2024-06-02T15:41:27.690254+0000","last_fullsized":"2024-06-02T15:41:27.690254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:56:31.597479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:55.648352+0000","last_change":"2024-06-02T15:38:59.601858+0000","last_active":"2024-06-02T15:40:55.648352+0000","last_peered":"2024-06-02T15:40:55.648352+0000","last_clean":"2024-06-02T15:40:55.648352+0000","last_became_active":"2024-06-02T15:38:59.601405+0000","last_became_peered":"2024-06-02T15:38:59.601405+0000","last_unstale":"2024-06-02T15:40:55.648352+0000","last_undegraded":"2024-06-02T15:40:55.648352+0000","last_fullsized":"2024-06-02T15:40:55.648352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:22:32.090925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":68,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849695+0000","last_change":"2024-06-02T15:38:59.191512+0000","last_active":"2024-06-02T15:40:20.849695+0000","last_peered":"2024-06-02T15:40:20.849695+0000","last_clean":"2024-06-02T15:40:20.849695+0000","last_became_active":"2024-06-02T15:38:59.185645+0000","last_became_peered":"2024-06-02T15:38:59.185645+0000","last_unstale":"2024-06-02T15:40:20.849695+0000","last_undegraded":"2024-06-02T15:40:20.849695+0000","last_fullsized":"2024-06-02T15:40:20.849695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:06.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705339+0000","last_change":"2024-06-02T15:38:16.478937+0000","last_active":"2024-06-02T15:41:27.705339+0000","last_peered":"2024-06-02T15:41:27.705339+0000","last_clean":"2024-06-02T15:41:27.705339+0000","last_became_active":"2024-06-02T15:38:16.478374+0000","last_became_peered":"2024-06-02T15:38:16.478374+0000","last_unstale":"2024-06-02T15:41:27.705339+0000","last_undegraded":"2024-06-02T15:41:27.705339+0000","last_fullsized":"2024-06-02T15:41:27.705339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:19.259111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.145879+0000","last_change":"2024-06-02T15:40:00.041841+0000","last_active":"2024-06-02T15:40:54.145879+0000","last_peered":"2024-06-02T15:40:54.145879+0000","last_clean":"2024-06-02T15:40:54.145879+0000","last_became_active":"2024-06-02T15:40:00.041393+0000","last_became_peered":"2024-06-02T15:40:00.041393+0000","last_unstale":"2024-06-02T15:40:54.145879+0000","last_undegraded":"2024-06-02T15:40:54.145879+0000","last_fullsized":"2024-06-02T15:40:54.145879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:22:48.250433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,10,3],"acting":[4,6,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:12.337384+0000","last_change":"2024-06-02T15:38:12.409108+0000","last_active":"2024-06-02T15:40:12.337384+0000","last_peered":"2024-06-02T15:40:12.337384+0000","last_clean":"2024-06-02T15:40:12.337384+0000","last_became_active":"2024-06-02T15:38:12.408435+0000","last_became_peered":"2024-06-02T15:38:12.408435+0000","last_unstale":"2024-06-02T15:40:12.337384+0000","last_undegraded":"2024-06-02T15:40:12.337384+0000","last_fullsized":"2024-06-02T15:40:12.337384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:41:02.031144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708471+0000","last_change":"2024-06-02T15:38:16.472717+0000","last_active":"2024-06-02T15:41:27.708471+0000","last_peered":"2024-06-02T15:41:27.708471+0000","last_clean":"2024-06-02T15:41:27.708471+0000","last_became_active":"2024-06-02T15:38:16.472340+0000","last_became_peered":"2024-06-02T15:38:16.472340+0000","last_unstale":"2024-06-02T15:41:27.708471+0000","last_undegraded":"2024-06-02T15:41:27.708471+0000","last_fullsized":"2024-06-02T15:41:27.708471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:45:24.370929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031438+0000","last_change":"2024-06-02T15:38:29.794115+0000","last_active":"2024-06-02T15:40:00.031438+0000","last_peered":"2024-06-02T15:40:00.031438+0000","last_clean":"2024-06-02T15:40:00.031438+0000","last_became_active":"2024-06-02T15:38:29.793695+0000","last_became_peered":"2024-06-02T15:38:29.793695+0000","last_unstale":"2024-06-02T15:40:00.031438+0000","last_undegraded":"2024-06-02T15:40:00.031438+0000","last_fullsized":"2024-06-02T15:40:00.031438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:16:52.644954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850859+0000","last_change":"2024-06-02T15:38:12.392778+0000","last_active":"2024-06-02T15:40:45.850859+0000","last_peered":"2024-06-02T15:40:45.850859+0000","last_clean":"2024-06-02T15:40:45.850859+0000","last_became_active":"2024-06-02T15:38:12.392419+0000","last_became_peered":"2024-06-02T15:38:12.392419+0000","last_unstale":"2024-06-02T15:40:45.850859+0000","last_undegraded":"2024-06-02T15:40:45.850859+0000","last_fullsized":"2024-06-02T15:40:45.850859+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:06:48.255766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697673+0000","last_change":"2024-06-02T15:38:16.483275+0000","last_active":"2024-06-02T15:41:27.697673+0000","last_peered":"2024-06-02T15:41:27.697673+0000","last_clean":"2024-06-02T15:41:27.697673+0000","last_became_active":"2024-06-02T15:38:16.482926+0000","last_became_peered":"2024-06-02T15:38:16.482926+0000","last_unstale":"2024-06-02T15:41:27.697673+0000","last_undegraded":"2024-06-02T15:41:27.697673+0000","last_fullsized":"2024-06-02T15:41:27.697673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:28:43.037171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031342+0000","last_change":"2024-06-02T15:38:29.780017+0000","last_active":"2024-06-02T15:40:00.031342+0000","last_peered":"2024-06-02T15:40:00.031342+0000","last_clean":"2024-06-02T15:40:00.031342+0000","last_became_active":"2024-06-02T15:38:29.779603+0000","last_became_peered":"2024-06-02T15:38:29.779603+0000","last_unstale":"2024-06-02T15:40:00.031342+0000","last_undegraded":"2024-06-02T15:40:00.031342+0000","last_fullsized":"2024-06-02T15:40:00.031342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:04.162874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.850848+0000","last_change":"2024-06-02T15:38:12.394798+0000","last_active":"2024-06-02T15:40:45.850848+0000","last_peered":"2024-06-02T15:40:45.850848+0000","last_clean":"2024-06-02T15:40:45.850848+0000","last_became_active":"2024-06-02T15:38:12.394365+0000","last_became_peered":"2024-06-02T15:38:12.394365+0000","last_unstale":"2024-06-02T15:40:45.850848+0000","last_undegraded":"2024-06-02T15:40:45.850848+0000","last_fullsized":"2024-06-02T15:40:45.850848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:05:58.869371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696163+0000","last_change":"2024-06-02T15:38:16.462195+0000","last_active":"2024-06-02T15:41:27.696163+0000","last_peered":"2024-06-02T15:41:27.696163+0000","last_clean":"2024-06-02T15:41:27.696163+0000","last_became_active":"2024-06-02T15:38:16.461830+0000","last_became_peered":"2024-06-02T15:38:16.461830+0000","last_unstale":"2024-06-02T15:41:27.696163+0000","last_undegraded":"2024-06-02T15:41:27.696163+0000","last_fullsized":"2024-06-02T15:41:27.696163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:49:47.464287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031260+0000","last_change":"2024-06-02T15:38:29.765746+0000","last_active":"2024-06-02T15:40:00.031260+0000","last_peered":"2024-06-02T15:40:00.031260+0000","last_clean":"2024-06-02T15:40:00.031260+0000","last_became_active":"2024-06-02T15:38:29.765201+0000","last_became_peered":"2024-06-02T15:38:29.765201+0000","last_unstale":"2024-06-02T15:40:00.031260+0000","last_undegraded":"2024-06-02T15:40:00.031260+0000","last_fullsized":"2024-06-02T15:40:00.031260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:58:22.311407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":280,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.466055+0000","last_change":"2024-06-02T15:38:12.392588+0000","last_active":"2024-06-02T15:43:17.466055+0000","last_peered":"2024-06-02T15:43:17.466055+0000","last_clean":"2024-06-02T15:43:17.466055+0000","last_became_active":"2024-06-02T15:38:12.392234+0000","last_became_peered":"2024-06-02T15:38:12.392234+0000","last_unstale":"2024-06-02T15:43:17.466055+0000","last_undegraded":"2024-06-02T15:43:17.466055+0000","last_fullsized":"2024-06-02T15:43:17.466055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:03.410268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":227,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.697597+0000","last_change":"2024-06-02T15:38:16.471712+0000","last_active":"2024-06-02T15:41:27.697597+0000","last_peered":"2024-06-02T15:41:27.697597+0000","last_clean":"2024-06-02T15:41:27.697597+0000","last_became_active":"2024-06-02T15:38:16.471351+0000","last_became_peered":"2024-06-02T15:38:16.471351+0000","last_unstale":"2024-06-02T15:41:27.697597+0000","last_undegraded":"2024-06-02T15:41:27.697597+0000","last_fullsized":"2024-06-02T15:41:27.697597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:45:11.082858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:00.031151+0000","last_change":"2024-06-02T15:38:29.792265+0000","last_active":"2024-06-02T15:40:00.031151+0000","last_peered":"2024-06-02T15:40:00.031151+0000","last_clean":"2024-06-02T15:40:00.031151+0000","last_became_active":"2024-06-02T15:38:29.791729+0000","last_became_peered":"2024-06-02T15:38:29.791729+0000","last_unstale":"2024-06-02T15:40:00.031151+0000","last_undegraded":"2024-06-02T15:40:00.031151+0000","last_fullsized":"2024-06-02T15:40:00.031151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:13:31.637426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:42.361651+0000","last_change":"2024-06-02T15:38:12.402403+0000","last_active":"2024-06-02T15:40:42.361651+0000","last_peered":"2024-06-02T15:40:42.361651+0000","last_clean":"2024-06-02T15:40:42.361651+0000","last_became_active":"2024-06-02T15:38:12.401978+0000","last_became_peered":"2024-06-02T15:38:12.401978+0000","last_unstale":"2024-06-02T15:40:42.361651+0000","last_undegraded":"2024-06-02T15:40:42.361651+0000","last_fullsized":"2024-06-02T15:40:42.361651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:33:42.325058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711391+0000","last_change":"2024-06-02T15:38:16.476630+0000","last_active":"2024-06-02T15:41:27.711391+0000","last_peered":"2024-06-02T15:41:27.711391+0000","last_clean":"2024-06-02T15:41:27.711391+0000","last_became_active":"2024-06-02T15:38:16.476355+0000","last_became_peered":"2024-06-02T15:38:16.476355+0000","last_unstale":"2024-06-02T15:41:27.711391+0000","last_undegraded":"2024-06-02T15:41:27.711391+0000","last_fullsized":"2024-06-02T15:41:27.711391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:31:56.239866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581825+0000","last_change":"2024-06-02T15:38:29.785901+0000","last_active":"2024-06-02T15:40:34.581825+0000","last_peered":"2024-06-02T15:40:34.581825+0000","last_clean":"2024-06-02T15:40:34.581825+0000","last_became_active":"2024-06-02T15:38:29.785365+0000","last_became_peered":"2024-06-02T15:38:29.785365+0000","last_unstale":"2024-06-02T15:40:34.581825+0000","last_undegraded":"2024-06-02T15:40:34.581825+0000","last_fullsized":"2024-06-02T15:40:34.581825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:43.093754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463524+0000","last_change":"2024-06-02T15:38:12.404942+0000","last_active":"2024-06-02T15:40:39.463524+0000","last_peered":"2024-06-02T15:40:39.463524+0000","last_clean":"2024-06-02T15:40:39.463524+0000","last_became_active":"2024-06-02T15:38:12.403771+0000","last_became_peered":"2024-06-02T15:38:12.403771+0000","last_unstale":"2024-06-02T15:40:39.463524+0000","last_undegraded":"2024-06-02T15:40:39.463524+0000","last_fullsized":"2024-06-02T15:40:39.463524+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:20:13.335950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711431+0000","last_change":"2024-06-02T15:38:16.485341+0000","last_active":"2024-06-02T15:41:27.711431+0000","last_peered":"2024-06-02T15:41:27.711431+0000","last_clean":"2024-06-02T15:41:27.711431+0000","last_became_active":"2024-06-02T15:38:16.484091+0000","last_became_peered":"2024-06-02T15:38:16.484091+0000","last_unstale":"2024-06-02T15:41:27.711431+0000","last_undegraded":"2024-06-02T15:41:27.711431+0000","last_fullsized":"2024-06-02T15:41:27.711431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:50.439653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:31.928337+0000","last_change":"2024-06-02T15:38:29.786019+0000","last_active":"2024-06-02T15:40:31.928337+0000","last_peered":"2024-06-02T15:40:31.928337+0000","last_clean":"2024-06-02T15:40:31.928337+0000","last_became_active":"2024-06-02T15:38:29.781035+0000","last_became_peered":"2024-06-02T15:38:29.781035+0000","last_unstale":"2024-06-02T15:40:31.928337+0000","last_undegraded":"2024-06-02T15:40:31.928337+0000","last_fullsized":"2024-06-02T15:40:31.928337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:42:16.763654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929241+0000","last_change":"2024-06-02T15:38:12.394704+0000","last_active":"2024-06-02T15:40:38.929241+0000","last_peered":"2024-06-02T15:40:38.929241+0000","last_clean":"2024-06-02T15:40:38.929241+0000","last_became_active":"2024-06-02T15:38:12.394321+0000","last_became_peered":"2024-06-02T15:38:12.394321+0000","last_unstale":"2024-06-02T15:40:38.929241+0000","last_undegraded":"2024-06-02T15:40:38.929241+0000","last_fullsized":"2024-06-02T15:40:38.929241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:31:27.514847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.710382+0000","last_change":"2024-06-02T15:38:16.479107+0000","last_active":"2024-06-02T15:41:27.710382+0000","last_peered":"2024-06-02T15:41:27.710382+0000","last_clean":"2024-06-02T15:41:27.710382+0000","last_became_active":"2024-06-02T15:38:16.478735+0000","last_became_peered":"2024-06-02T15:38:16.478735+0000","last_unstale":"2024-06-02T15:41:27.710382+0000","last_undegraded":"2024-06-02T15:41:27.710382+0000","last_fullsized":"2024-06-02T15:41:27.710382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:54.637401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.446626+0000","last_change":"2024-06-02T15:38:29.779398+0000","last_active":"2024-06-02T15:40:50.446626+0000","last_peered":"2024-06-02T15:40:50.446626+0000","last_clean":"2024-06-02T15:40:50.446626+0000","last_became_active":"2024-06-02T15:38:29.778981+0000","last_became_peered":"2024-06-02T15:38:29.778981+0000","last_unstale":"2024-06-02T15:40:50.446626+0000","last_undegraded":"2024-06-02T15:40:50.446626+0000","last_fullsized":"2024-06-02T15:40:50.446626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:00:13.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":276,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:20.470554+0000","last_change":"2024-06-02T15:38:12.407443+0000","last_active":"2024-06-02T15:43:20.470554+0000","last_peered":"2024-06-02T15:43:20.470554+0000","last_clean":"2024-06-02T15:43:20.470554+0000","last_became_active":"2024-06-02T15:38:12.407091+0000","last_became_peered":"2024-06-02T15:38:12.407091+0000","last_unstale":"2024-06-02T15:43:20.470554+0000","last_undegraded":"2024-06-02T15:43:20.470554+0000","last_fullsized":"2024-06-02T15:43:20.470554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:37:24.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701105+0000","last_change":"2024-06-02T15:38:16.467856+0000","last_active":"2024-06-02T15:41:27.701105+0000","last_peered":"2024-06-02T15:41:27.701105+0000","last_clean":"2024-06-02T15:41:27.701105+0000","last_became_active":"2024-06-02T15:38:16.467519+0000","last_became_peered":"2024-06-02T15:38:16.467519+0000","last_unstale":"2024-06-02T15:41:27.701105+0000","last_undegraded":"2024-06-02T15:41:27.701105+0000","last_fullsized":"2024-06-02T15:41:27.701105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:30:34.599144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854789+0000","last_change":"2024-06-02T15:38:29.796580+0000","last_active":"2024-06-02T15:40:50.854789+0000","last_peered":"2024-06-02T15:40:50.854789+0000","last_clean":"2024-06-02T15:40:50.854789+0000","last_became_active":"2024-06-02T15:38:29.795347+0000","last_became_peered":"2024-06-02T15:38:29.795347+0000","last_unstale":"2024-06-02T15:40:50.854789+0000","last_undegraded":"2024-06-02T15:40:50.854789+0000","last_fullsized":"2024-06-02T15:40:50.854789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:41:39.138210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:40.421522+0000","last_change":"2024-06-02T15:38:12.396852+0000","last_active":"2024-06-02T15:40:40.421522+0000","last_peered":"2024-06-02T15:40:40.421522+0000","last_clean":"2024-06-02T15:40:40.421522+0000","last_became_active":"2024-06-02T15:38:12.396401+0000","last_became_peered":"2024-06-02T15:38:12.396401+0000","last_unstale":"2024-06-02T15:40:40.421522+0000","last_undegraded":"2024-06-02T15:40:40.421522+0000","last_fullsized":"2024-06-02T15:40:40.421522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:42:43.284207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706943+0000","last_change":"2024-06-02T15:38:16.461926+0000","last_active":"2024-06-02T15:41:27.706943+0000","last_peered":"2024-06-02T15:41:27.706943+0000","last_clean":"2024-06-02T15:41:27.706943+0000","last_became_active":"2024-06-02T15:38:16.461543+0000","last_became_peered":"2024-06-02T15:38:16.461543+0000","last_unstale":"2024-06-02T15:41:27.706943+0000","last_undegraded":"2024-06-02T15:41:27.706943+0000","last_fullsized":"2024-06-02T15:41:27.706943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:47.938547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.145181+0000","last_change":"2024-06-02T15:38:29.794422+0000","last_active":"2024-06-02T15:40:49.145181+0000","last_peered":"2024-06-02T15:40:49.145181+0000","last_clean":"2024-06-02T15:40:49.145181+0000","last_became_active":"2024-06-02T15:38:29.793974+0000","last_became_peered":"2024-06-02T15:38:29.793974+0000","last_unstale":"2024-06-02T15:40:49.145181+0000","last_undegraded":"2024-06-02T15:40:49.145181+0000","last_fullsized":"2024-06-02T15:40:49.145181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:28.053087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929293+0000","last_change":"2024-06-02T15:38:12.400464+0000","last_active":"2024-06-02T15:40:38.929293+0000","last_peered":"2024-06-02T15:40:38.929293+0000","last_clean":"2024-06-02T15:40:38.929293+0000","last_became_active":"2024-06-02T15:38:12.400099+0000","last_became_peered":"2024-06-02T15:38:12.400099+0000","last_unstale":"2024-06-02T15:40:38.929293+0000","last_undegraded":"2024-06-02T15:40:38.929293+0000","last_fullsized":"2024-06-02T15:40:38.929293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:46:01.245197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698278+0000","last_change":"2024-06-02T15:38:16.467495+0000","last_active":"2024-06-02T15:41:27.698278+0000","last_peered":"2024-06-02T15:41:27.698278+0000","last_clean":"2024-06-02T15:41:27.698278+0000","last_became_active":"2024-06-02T15:38:16.467104+0000","last_became_peered":"2024-06-02T15:38:16.467104+0000","last_unstale":"2024-06-02T15:41:27.698278+0000","last_undegraded":"2024-06-02T15:41:27.698278+0000","last_fullsized":"2024-06-02T15:41:27.698278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:33:58.397061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:50.854862+0000","last_change":"2024-06-02T15:38:29.793586+0000","last_active":"2024-06-02T15:40:50.854862+0000","last_peered":"2024-06-02T15:40:50.854862+0000","last_clean":"2024-06-02T15:40:50.854862+0000","last_became_active":"2024-06-02T15:38:29.793112+0000","last_became_peered":"2024-06-02T15:38:29.793112+0000","last_unstale":"2024-06-02T15:40:50.854862+0000","last_undegraded":"2024-06-02T15:40:50.854862+0000","last_fullsized":"2024-06-02T15:40:50.854862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:56:14.365901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":566,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:21.472282+0000","last_change":"2024-06-02T15:38:59.940721+0000","last_active":"2024-06-02T15:43:21.472282+0000","last_peered":"2024-06-02T15:43:21.472282+0000","last_clean":"2024-06-02T15:43:21.472282+0000","last_became_active":"2024-06-02T15:38:59.933565+0000","last_became_peered":"2024-06-02T15:38:59.933565+0000","last_unstale":"2024-06-02T15:43:21.472282+0000","last_undegraded":"2024-06-02T15:43:21.472282+0000","last_fullsized":"2024-06-02T15:43:21.472282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:19:11.528490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69988,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":464,"num_read_kb":301,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685871+0000","last_change":"2024-06-02T15:38:16.459848+0000","last_active":"2024-06-02T15:41:27.685871+0000","last_peered":"2024-06-02T15:41:27.685871+0000","last_clean":"2024-06-02T15:41:27.685871+0000","last_became_active":"2024-06-02T15:38:16.459504+0000","last_became_peered":"2024-06-02T15:38:16.459504+0000","last_unstale":"2024-06-02T15:41:27.685871+0000","last_undegraded":"2024-06-02T15:41:27.685871+0000","last_fullsized":"2024-06-02T15:41:27.685871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:58:48.361095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.931198+0000","last_change":"2024-06-02T15:38:29.779436+0000","last_active":"2024-06-02T15:40:53.931198+0000","last_peered":"2024-06-02T15:40:53.931198+0000","last_clean":"2024-06-02T15:40:53.931198+0000","last_became_active":"2024-06-02T15:38:29.778991+0000","last_became_peered":"2024-06-02T15:38:29.778991+0000","last_unstale":"2024-06-02T15:40:53.931198+0000","last_undegraded":"2024-06-02T15:40:53.931198+0000","last_fullsized":"2024-06-02T15:40:53.931198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:51:12.233727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646104+0000","last_change":"2024-06-02T15:38:12.401071+0000","last_active":"2024-06-02T15:40:35.646104+0000","last_peered":"2024-06-02T15:40:35.646104+0000","last_clean":"2024-06-02T15:40:35.646104+0000","last_became_active":"2024-06-02T15:38:12.400667+0000","last_became_peered":"2024-06-02T15:38:12.400667+0000","last_unstale":"2024-06-02T15:40:35.646104+0000","last_undegraded":"2024-06-02T15:40:35.646104+0000","last_fullsized":"2024-06-02T15:40:35.646104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:03:32.219903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711870+0000","last_change":"2024-06-02T15:38:16.479146+0000","last_active":"2024-06-02T15:41:27.711870+0000","last_peered":"2024-06-02T15:41:27.711870+0000","last_clean":"2024-06-02T15:41:27.711870+0000","last_became_active":"2024-06-02T15:38:16.477525+0000","last_became_peered":"2024-06-02T15:38:16.477525+0000","last_unstale":"2024-06-02T15:41:27.711870+0000","last_undegraded":"2024-06-02T15:41:27.711870+0000","last_fullsized":"2024-06-02T15:41:27.711870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:20:59.239936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:49.583429+0000","last_change":"2024-06-02T15:38:29.786118+0000","last_active":"2024-06-02T15:40:49.583429+0000","last_peered":"2024-06-02T15:40:49.583429+0000","last_clean":"2024-06-02T15:40:49.583429+0000","last_became_active":"2024-06-02T15:38:29.785513+0000","last_became_peered":"2024-06-02T15:38:29.785513+0000","last_unstale":"2024-06-02T15:40:49.583429+0000","last_undegraded":"2024-06-02T15:40:49.583429+0000","last_fullsized":"2024-06-02T15:40:49.583429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:01:23.691669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:33.928487+0000","last_change":"2024-06-02T15:38:12.403369+0000","last_active":"2024-06-02T15:40:33.928487+0000","last_peered":"2024-06-02T15:40:33.928487+0000","last_clean":"2024-06-02T15:40:33.928487+0000","last_became_active":"2024-06-02T15:38:12.403001+0000","last_became_peered":"2024-06-02T15:38:12.403001+0000","last_unstale":"2024-06-02T15:40:33.928487+0000","last_undegraded":"2024-06-02T15:40:33.928487+0000","last_fullsized":"2024-06-02T15:40:33.928487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:28:57.373472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.696588+0000","last_change":"2024-06-02T15:38:16.484454+0000","last_active":"2024-06-02T15:41:27.696588+0000","last_peered":"2024-06-02T15:41:27.696588+0000","last_clean":"2024-06-02T15:41:27.696588+0000","last_became_active":"2024-06-02T15:38:16.484132+0000","last_became_peered":"2024-06-02T15:38:16.484132+0000","last_unstale":"2024-06-02T15:41:27.696588+0000","last_undegraded":"2024-06-02T15:41:27.696588+0000","last_fullsized":"2024-06-02T15:41:27.696588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:35:30.580237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:53.911547+0000","last_change":"2024-06-02T15:38:29.797793+0000","last_active":"2024-06-02T15:40:53.911547+0000","last_peered":"2024-06-02T15:40:53.911547+0000","last_clean":"2024-06-02T15:40:53.911547+0000","last_became_active":"2024-06-02T15:38:29.795928+0000","last_became_peered":"2024-06-02T15:38:29.795928+0000","last_unstale":"2024-06-02T15:40:53.911547+0000","last_undegraded":"2024-06-02T15:40:53.911547+0000","last_fullsized":"2024-06-02T15:40:53.911547+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:46:54.829723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":278,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615898+0000","last_change":"2024-06-02T15:38:12.394005+0000","last_active":"2024-06-02T15:43:17.615898+0000","last_peered":"2024-06-02T15:43:17.615898+0000","last_clean":"2024-06-02T15:43:17.615898+0000","last_became_active":"2024-06-02T15:38:12.393627+0000","last_became_peered":"2024-06-02T15:38:12.393627+0000","last_unstale":"2024-06-02T15:43:17.615898+0000","last_undegraded":"2024-06-02T15:43:17.615898+0000","last_fullsized":"2024-06-02T15:43:17.615898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:45:27.122483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687560+0000","last_change":"2024-06-02T15:38:16.468731+0000","last_active":"2024-06-02T15:41:27.687560+0000","last_peered":"2024-06-02T15:41:27.687560+0000","last_clean":"2024-06-02T15:41:27.687560+0000","last_became_active":"2024-06-02T15:38:16.468388+0000","last_became_peered":"2024-06-02T15:38:16.468388+0000","last_unstale":"2024-06-02T15:41:27.687560+0000","last_undegraded":"2024-06-02T15:41:27.687560+0000","last_fullsized":"2024-06-02T15:41:27.687560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:10:50.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:54.466042+0000","last_change":"2024-06-02T15:38:29.797928+0000","last_active":"2024-06-02T15:40:54.466042+0000","last_peered":"2024-06-02T15:40:54.466042+0000","last_clean":"2024-06-02T15:40:54.466042+0000","last_became_active":"2024-06-02T15:38:29.797395+0000","last_became_peered":"2024-06-02T15:38:29.797395+0000","last_unstale":"2024-06-02T15:40:54.466042+0000","last_undegraded":"2024-06-02T15:40:54.466042+0000","last_fullsized":"2024-06-02T15:40:54.466042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:44:15.662294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849838+0000","last_change":"2024-06-02T15:38:12.396608+0000","last_active":"2024-06-02T15:40:35.849838+0000","last_peered":"2024-06-02T15:40:35.849838+0000","last_clean":"2024-06-02T15:40:35.849838+0000","last_became_active":"2024-06-02T15:38:12.396209+0000","last_became_peered":"2024-06-02T15:38:12.396209+0000","last_unstale":"2024-06-02T15:40:35.849838+0000","last_undegraded":"2024-06-02T15:40:35.849838+0000","last_fullsized":"2024-06-02T15:40:35.849838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:31:04.438145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694458+0000","last_change":"2024-06-02T15:38:16.464546+0000","last_active":"2024-06-02T15:41:27.694458+0000","last_peered":"2024-06-02T15:41:27.694458+0000","last_clean":"2024-06-02T15:41:27.694458+0000","last_became_active":"2024-06-02T15:38:16.463200+0000","last_became_peered":"2024-06-02T15:38:16.463200+0000","last_unstale":"2024-06-02T15:41:27.694458+0000","last_undegraded":"2024-06-02T15:41:27.694458+0000","last_fullsized":"2024-06-02T15:41:27.694458+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:35:04.311297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929257+0000","last_change":"2024-06-02T15:38:29.779478+0000","last_active":"2024-06-02T15:40:38.929257+0000","last_peered":"2024-06-02T15:40:38.929257+0000","last_clean":"2024-06-02T15:40:38.929257+0000","last_became_active":"2024-06-02T15:38:29.779021+0000","last_became_peered":"2024-06-02T15:38:29.779021+0000","last_unstale":"2024-06-02T15:40:38.929257+0000","last_undegraded":"2024-06-02T15:40:38.929257+0000","last_fullsized":"2024-06-02T15:40:38.929257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:03:43.589400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849304+0000","last_change":"2024-06-02T15:38:12.390792+0000","last_active":"2024-06-02T15:40:30.849304+0000","last_peered":"2024-06-02T15:40:30.849304+0000","last_clean":"2024-06-02T15:40:30.849304+0000","last_became_active":"2024-06-02T15:38:12.390420+0000","last_became_peered":"2024-06-02T15:38:12.390420+0000","last_unstale":"2024-06-02T15:40:30.849304+0000","last_undegraded":"2024-06-02T15:40:30.849304+0000","last_fullsized":"2024-06-02T15:40:30.849304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:06:52.503691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.695493+0000","last_change":"2024-06-02T15:38:16.468721+0000","last_active":"2024-06-02T15:41:27.695493+0000","last_peered":"2024-06-02T15:41:27.695493+0000","last_clean":"2024-06-02T15:41:27.695493+0000","last_became_active":"2024-06-02T15:38:16.468378+0000","last_became_peered":"2024-06-02T15:38:16.468378+0000","last_unstale":"2024-06-02T15:41:27.695493+0000","last_undegraded":"2024-06-02T15:41:27.695493+0000","last_fullsized":"2024-06-02T15:41:27.695493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:40:48.395243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.849864+0000","last_change":"2024-06-02T15:38:29.786017+0000","last_active":"2024-06-02T15:40:35.849864+0000","last_peered":"2024-06-02T15:40:35.849864+0000","last_clean":"2024-06-02T15:40:35.849864+0000","last_became_active":"2024-06-02T15:38:29.785460+0000","last_became_peered":"2024-06-02T15:38:29.785460+0000","last_unstale":"2024-06-02T15:40:35.849864+0000","last_undegraded":"2024-06-02T15:40:35.849864+0000","last_fullsized":"2024-06-02T15:40:35.849864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:47:52.297686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.849635+0000","last_change":"2024-06-02T15:38:12.390894+0000","last_active":"2024-06-02T15:40:20.849635+0000","last_peered":"2024-06-02T15:40:20.849635+0000","last_clean":"2024-06-02T15:40:20.849635+0000","last_became_active":"2024-06-02T15:38:12.390411+0000","last_became_peered":"2024-06-02T15:38:12.390411+0000","last_unstale":"2024-06-02T15:40:20.849635+0000","last_undegraded":"2024-06-02T15:40:20.849635+0000","last_fullsized":"2024-06-02T15:40:20.849635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:53:39.622806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693299+0000","last_change":"2024-06-02T15:38:16.482062+0000","last_active":"2024-06-02T15:41:27.693299+0000","last_peered":"2024-06-02T15:41:27.693299+0000","last_clean":"2024-06-02T15:41:27.693299+0000","last_became_active":"2024-06-02T15:38:16.481746+0000","last_became_peered":"2024-06-02T15:38:16.481746+0000","last_unstale":"2024-06-02T15:41:27.693299+0000","last_undegraded":"2024-06-02T15:41:27.693299+0000","last_fullsized":"2024-06-02T15:41:27.693299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:41:41.029639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.462890+0000","last_change":"2024-06-02T15:38:29.782887+0000","last_active":"2024-06-02T15:40:34.462890+0000","last_peered":"2024-06-02T15:40:34.462890+0000","last_clean":"2024-06-02T15:40:34.462890+0000","last_became_active":"2024-06-02T15:38:29.782447+0000","last_became_peered":"2024-06-02T15:38:29.782447+0000","last_unstale":"2024-06-02T15:40:34.462890+0000","last_undegraded":"2024-06-02T15:40:34.462890+0000","last_fullsized":"2024-06-02T15:40:34.462890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:45:44.463198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.419076+0000","last_change":"2024-06-02T15:38:12.402518+0000","last_active":"2024-06-02T15:40:20.419076+0000","last_peered":"2024-06-02T15:40:20.419076+0000","last_clean":"2024-06-02T15:40:20.419076+0000","last_became_active":"2024-06-02T15:38:12.401999+0000","last_became_peered":"2024-06-02T15:38:12.401999+0000","last_unstale":"2024-06-02T15:40:20.419076+0000","last_undegraded":"2024-06-02T15:40:20.419076+0000","last_fullsized":"2024-06-02T15:40:20.419076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:15:43.354285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.676935+0000","last_change":"2024-06-02T15:38:16.475156+0000","last_active":"2024-06-02T15:41:27.676935+0000","last_peered":"2024-06-02T15:41:27.676935+0000","last_clean":"2024-06-02T15:41:27.676935+0000","last_became_active":"2024-06-02T15:38:16.474778+0000","last_became_peered":"2024-06-02T15:38:16.474778+0000","last_unstale":"2024-06-02T15:41:27.676935+0000","last_undegraded":"2024-06-02T15:41:27.676935+0000","last_fullsized":"2024-06-02T15:41:27.676935+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:30:03.965599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:39.463609+0000","last_change":"2024-06-02T15:38:29.781432+0000","last_active":"2024-06-02T15:40:39.463609+0000","last_peered":"2024-06-02T15:40:39.463609+0000","last_clean":"2024-06-02T15:40:39.463609+0000","last_became_active":"2024-06-02T15:38:29.780926+0000","last_became_peered":"2024-06-02T15:38:29.780926+0000","last_unstale":"2024-06-02T15:40:39.463609+0000","last_undegraded":"2024-06-02T15:40:39.463609+0000","last_fullsized":"2024-06-02T15:40:39.463609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:25.643876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.690130+0000","last_change":"2024-06-02T15:38:16.461168+0000","last_active":"2024-06-02T15:41:27.690130+0000","last_peered":"2024-06-02T15:41:27.690130+0000","last_clean":"2024-06-02T15:41:27.690130+0000","last_became_active":"2024-06-02T15:38:16.460843+0000","last_became_peered":"2024-06-02T15:38:16.460843+0000","last_unstale":"2024-06-02T15:41:27.690130+0000","last_undegraded":"2024-06-02T15:41:27.690130+0000","last_fullsized":"2024-06-02T15:41:27.690130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:14:35.464079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":274,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:43:17.615195+0000","last_change":"2024-06-02T15:38:12.402601+0000","last_active":"2024-06-02T15:43:17.615195+0000","last_peered":"2024-06-02T15:43:17.615195+0000","last_clean":"2024-06-02T15:43:17.615195+0000","last_became_active":"2024-06-02T15:38:12.401832+0000","last_became_peered":"2024-06-02T15:38:12.401832+0000","last_unstale":"2024-06-02T15:43:17.615195+0000","last_undegraded":"2024-06-02T15:43:17.615195+0000","last_fullsized":"2024-06-02T15:43:17.615195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:57:47.974156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":223,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.420757+0000","last_change":"2024-06-02T15:40:00.045344+0000","last_active":"2024-06-02T15:40:35.420757+0000","last_peered":"2024-06-02T15:40:35.420757+0000","last_clean":"2024-06-02T15:40:35.420757+0000","last_became_active":"2024-06-02T15:40:00.038967+0000","last_became_peered":"2024-06-02T15:40:00.038967+0000","last_unstale":"2024-06-02T15:40:35.420757+0000","last_undegraded":"2024-06-02T15:40:35.420757+0000","last_fullsized":"2024-06-02T15:40:35.420757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:24:03.155331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.708881+0000","last_change":"2024-06-02T15:38:16.474868+0000","last_active":"2024-06-02T15:41:27.708881+0000","last_peered":"2024-06-02T15:41:27.708881+0000","last_clean":"2024-06-02T15:41:27.708881+0000","last_became_active":"2024-06-02T15:38:16.472131+0000","last_became_peered":"2024-06-02T15:38:16.472131+0000","last_unstale":"2024-06-02T15:41:27.708881+0000","last_undegraded":"2024-06-02T15:41:27.708881+0000","last_fullsized":"2024-06-02T15:41:27.708881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:39:09.845848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:19.142002+0000","last_change":"2024-06-02T15:38:12.405803+0000","last_active":"2024-06-02T15:40:19.142002+0000","last_peered":"2024-06-02T15:40:19.142002+0000","last_clean":"2024-06-02T15:40:19.142002+0000","last_became_active":"2024-06-02T15:38:12.405429+0000","last_became_peered":"2024-06-02T15:38:12.405429+0000","last_unstale":"2024-06-02T15:40:19.142002+0000","last_undegraded":"2024-06-02T15:40:19.142002+0000","last_fullsized":"2024-06-02T15:40:19.142002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:56:40.059675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:10.424476+0000","last_change":"2024-06-02T15:36:34.697680+0000","last_active":"2024-06-02T15:41:10.424476+0000","last_peered":"2024-06-02T15:41:10.424476+0000","last_clean":"2024-06-02T15:41:10.424476+0000","last_became_active":"2024-06-02T15:36:34.693040+0000","last_became_peered":"2024-06-02T15:36:34.693040+0000","last_unstale":"2024-06-02T15:41:10.424476+0000","last_undegraded":"2024-06-02T15:41:10.424476+0000","last_fullsized":"2024-06-02T15:41:10.424476+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:32:42.629792+0000","last_clean_scrub_stamp":"2024-06-02T15:32:42.629792+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:20.020789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929309+0000","last_change":"2024-06-02T15:38:29.788091+0000","last_active":"2024-06-02T15:40:38.929309+0000","last_peered":"2024-06-02T15:40:38.929309+0000","last_clean":"2024-06-02T15:40:38.929309+0000","last_became_active":"2024-06-02T15:38:29.787682+0000","last_became_peered":"2024-06-02T15:38:29.787682+0000","last_unstale":"2024-06-02T15:40:38.929309+0000","last_undegraded":"2024-06-02T15:40:38.929309+0000","last_fullsized":"2024-06-02T15:40:38.929309+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:37:50.494653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685503+0000","last_change":"2024-06-02T15:38:16.475161+0000","last_active":"2024-06-02T15:41:27.685503+0000","last_peered":"2024-06-02T15:41:27.685503+0000","last_clean":"2024-06-02T15:41:27.685503+0000","last_became_active":"2024-06-02T15:38:16.468889+0000","last_became_peered":"2024-06-02T15:38:16.468889+0000","last_unstale":"2024-06-02T15:41:27.685503+0000","last_undegraded":"2024-06-02T15:41:27.685503+0000","last_fullsized":"2024-06-02T15:41:27.685503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:59:34.111440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:15.418582+0000","last_change":"2024-06-02T15:38:59.933808+0000","last_active":"2024-06-02T15:40:15.418582+0000","last_peered":"2024-06-02T15:40:15.418582+0000","last_clean":"2024-06-02T15:40:15.418582+0000","last_became_active":"2024-06-02T15:38:59.933223+0000","last_became_peered":"2024-06-02T15:38:59.933223+0000","last_unstale":"2024-06-02T15:40:15.418582+0000","last_undegraded":"2024-06-02T15:40:15.418582+0000","last_fullsized":"2024-06-02T15:40:15.418582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:37.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:34.581781+0000","last_change":"2024-06-02T15:38:29.782959+0000","last_active":"2024-06-02T15:40:34.581781+0000","last_peered":"2024-06-02T15:40:34.581781+0000","last_clean":"2024-06-02T15:40:34.581781+0000","last_became_active":"2024-06-02T15:38:29.782498+0000","last_became_peered":"2024-06-02T15:38:29.782498+0000","last_unstale":"2024-06-02T15:40:34.581781+0000","last_undegraded":"2024-06-02T15:40:34.581781+0000","last_fullsized":"2024-06-02T15:40:34.581781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:29:45.643260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:20.848207+0000","last_change":"2024-06-02T15:38:12.403520+0000","last_active":"2024-06-02T15:40:20.848207+0000","last_peered":"2024-06-02T15:40:20.848207+0000","last_clean":"2024-06-02T15:40:20.848207+0000","last_became_active":"2024-06-02T15:38:12.403081+0000","last_became_peered":"2024-06-02T15:38:12.403081+0000","last_unstale":"2024-06-02T15:40:20.848207+0000","last_undegraded":"2024-06-02T15:40:20.848207+0000","last_fullsized":"2024-06-02T15:40:20.848207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:20:04.348406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.693951+0000","last_change":"2024-06-02T15:38:16.487638+0000","last_active":"2024-06-02T15:41:27.693951+0000","last_peered":"2024-06-02T15:41:27.693951+0000","last_clean":"2024-06-02T15:41:27.693951+0000","last_became_active":"2024-06-02T15:38:16.485529+0000","last_became_peered":"2024-06-02T15:38:16.485529+0000","last_unstale":"2024-06-02T15:41:27.693951+0000","last_undegraded":"2024-06-02T15:41:27.693951+0000","last_fullsized":"2024-06-02T15:41:27.693951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:28:38.122222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:32.340299+0000","last_change":"2024-06-02T15:38:29.791106+0000","last_active":"2024-06-02T15:40:32.340299+0000","last_peered":"2024-06-02T15:40:32.340299+0000","last_clean":"2024-06-02T15:40:32.340299+0000","last_became_active":"2024-06-02T15:38:29.790613+0000","last_became_peered":"2024-06-02T15:38:29.790613+0000","last_unstale":"2024-06-02T15:40:32.340299+0000","last_undegraded":"2024-06-02T15:40:32.340299+0000","last_fullsized":"2024-06-02T15:40:32.340299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:47:48.938382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:42:35.870244+0000","last_change":"2024-06-02T15:38:12.407659+0000","last_active":"2024-06-02T15:42:35.870244+0000","last_peered":"2024-06-02T15:42:35.870244+0000","last_clean":"2024-06-02T15:42:35.870244+0000","last_became_active":"2024-06-02T15:38:12.406966+0000","last_became_peered":"2024-06-02T15:38:12.406966+0000","last_unstale":"2024-06-02T15:42:35.870244+0000","last_undegraded":"2024-06-02T15:42:35.870244+0000","last_fullsized":"2024-06-02T15:42:35.870244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:00:35.319290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.709823+0000","last_change":"2024-06-02T15:38:16.473592+0000","last_active":"2024-06-02T15:41:27.709823+0000","last_peered":"2024-06-02T15:41:27.709823+0000","last_clean":"2024-06-02T15:41:27.709823+0000","last_became_active":"2024-06-02T15:38:16.471664+0000","last_became_peered":"2024-06-02T15:38:16.471664+0000","last_unstale":"2024-06-02T15:41:27.709823+0000","last_undegraded":"2024-06-02T15:41:27.709823+0000","last_fullsized":"2024-06-02T15:41:27.709823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:11:23.248032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:35.646076+0000","last_change":"2024-06-02T15:38:29.763181+0000","last_active":"2024-06-02T15:40:35.646076+0000","last_peered":"2024-06-02T15:40:35.646076+0000","last_clean":"2024-06-02T15:40:35.646076+0000","last_became_active":"2024-06-02T15:38:29.762744+0000","last_became_peered":"2024-06-02T15:38:29.762744+0000","last_unstale":"2024-06-02T15:40:35.646076+0000","last_undegraded":"2024-06-02T15:40:35.646076+0000","last_fullsized":"2024-06-02T15:40:35.646076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:03:05.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:24.142446+0000","last_change":"2024-06-02T15:38:12.517472+0000","last_active":"2024-06-02T15:40:24.142446+0000","last_peered":"2024-06-02T15:40:24.142446+0000","last_clean":"2024-06-02T15:40:24.142446+0000","last_became_active":"2024-06-02T15:38:12.517112+0000","last_became_peered":"2024-06-02T15:38:12.517112+0000","last_unstale":"2024-06-02T15:40:24.142446+0000","last_undegraded":"2024-06-02T15:40:24.142446+0000","last_fullsized":"2024-06-02T15:40:24.142446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:52:19.744241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711979+0000","last_change":"2024-06-02T15:39:00.133571+0000","last_active":"2024-06-02T15:41:27.711979+0000","last_peered":"2024-06-02T15:41:27.711979+0000","last_clean":"2024-06-02T15:41:27.711979+0000","last_became_active":"2024-06-02T15:39:00.132518+0000","last_became_peered":"2024-06-02T15:39:00.132518+0000","last_unstale":"2024-06-02T15:41:27.711979+0000","last_undegraded":"2024-06-02T15:41:27.711979+0000","last_fullsized":"2024-06-02T15:41:27.711979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:43:24.796946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144738+0000","last_change":"2024-06-02T15:38:29.796740+0000","last_active":"2024-06-02T15:40:44.144738+0000","last_peered":"2024-06-02T15:40:44.144738+0000","last_clean":"2024-06-02T15:40:44.144738+0000","last_became_active":"2024-06-02T15:38:29.796289+0000","last_became_peered":"2024-06-02T15:38:29.796289+0000","last_unstale":"2024-06-02T15:40:44.144738+0000","last_undegraded":"2024-06-02T15:40:44.144738+0000","last_fullsized":"2024-06-02T15:40:44.144738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:23.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:26.381402+0000","last_change":"2024-06-02T15:38:12.396941+0000","last_active":"2024-06-02T15:40:26.381402+0000","last_peered":"2024-06-02T15:40:26.381402+0000","last_clean":"2024-06-02T15:40:26.381402+0000","last_became_active":"2024-06-02T15:38:12.396596+0000","last_became_peered":"2024-06-02T15:38:12.396596+0000","last_unstale":"2024-06-02T15:40:26.381402+0000","last_undegraded":"2024-06-02T15:40:26.381402+0000","last_fullsized":"2024-06-02T15:40:26.381402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:38:15.670288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.680443+0000","last_change":"2024-06-02T15:38:16.459936+0000","last_active":"2024-06-02T15:41:27.680443+0000","last_peered":"2024-06-02T15:41:27.680443+0000","last_clean":"2024-06-02T15:41:27.680443+0000","last_became_active":"2024-06-02T15:38:16.459559+0000","last_became_peered":"2024-06-02T15:38:16.459559+0000","last_unstale":"2024-06-02T15:41:27.680443+0000","last_undegraded":"2024-06-02T15:41:27.680443+0000","last_fullsized":"2024-06-02T15:41:27.680443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:17:02.594505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:45.853966+0000","last_change":"2024-06-02T15:38:29.797399+0000","last_active":"2024-06-02T15:40:45.853966+0000","last_peered":"2024-06-02T15:40:45.853966+0000","last_clean":"2024-06-02T15:40:45.853966+0000","last_became_active":"2024-06-02T15:38:29.794609+0000","last_became_peered":"2024-06-02T15:38:29.794609+0000","last_unstale":"2024-06-02T15:40:45.853966+0000","last_undegraded":"2024-06-02T15:40:45.853966+0000","last_fullsized":"2024-06-02T15:40:45.853966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:41:07.339276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:22.338983+0000","last_change":"2024-06-02T15:38:12.404771+0000","last_active":"2024-06-02T15:40:22.338983+0000","last_peered":"2024-06-02T15:40:22.338983+0000","last_clean":"2024-06-02T15:40:22.338983+0000","last_became_active":"2024-06-02T15:38:12.404440+0000","last_became_peered":"2024-06-02T15:38:12.404440+0000","last_unstale":"2024-06-02T15:40:22.338983+0000","last_undegraded":"2024-06-02T15:40:22.338983+0000","last_fullsized":"2024-06-02T15:40:22.338983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:04:44.313018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703948+0000","last_change":"2024-06-02T15:38:16.472814+0000","last_active":"2024-06-02T15:41:27.703948+0000","last_peered":"2024-06-02T15:41:27.703948+0000","last_clean":"2024-06-02T15:41:27.703948+0000","last_became_active":"2024-06-02T15:38:16.472363+0000","last_became_peered":"2024-06-02T15:38:16.472363+0000","last_unstale":"2024-06-02T15:41:27.703948+0000","last_undegraded":"2024-06-02T15:41:27.703948+0000","last_fullsized":"2024-06-02T15:41:27.703948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:33:47.361030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144717+0000","last_change":"2024-06-02T15:38:29.794485+0000","last_active":"2024-06-02T15:40:44.144717+0000","last_peered":"2024-06-02T15:40:44.144717+0000","last_clean":"2024-06-02T15:40:44.144717+0000","last_became_active":"2024-06-02T15:38:29.793619+0000","last_became_peered":"2024-06-02T15:38:29.793619+0000","last_unstale":"2024-06-02T15:40:44.144717+0000","last_undegraded":"2024-06-02T15:40:44.144717+0000","last_fullsized":"2024-06-02T15:40:44.144717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:14:44.192663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:09.585558+0000","last_change":"2024-06-02T15:38:59.601991+0000","last_active":"2024-06-02T15:41:09.585558+0000","last_peered":"2024-06-02T15:41:09.585558+0000","last_clean":"2024-06-02T15:41:09.585558+0000","last_became_active":"2024-06-02T15:38:59.601553+0000","last_became_peered":"2024-06-02T15:38:59.601553+0000","last_unstale":"2024-06-02T15:41:09.585558+0000","last_undegraded":"2024-06-02T15:41:09.585558+0000","last_fullsized":"2024-06-02T15:41:09.585558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:08:13.557743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711497+0000","last_change":"2024-06-02T15:38:16.484225+0000","last_active":"2024-06-02T15:41:27.711497+0000","last_peered":"2024-06-02T15:41:27.711497+0000","last_clean":"2024-06-02T15:41:27.711497+0000","last_became_active":"2024-06-02T15:38:16.483676+0000","last_became_peered":"2024-06-02T15:38:16.483676+0000","last_unstale":"2024-06-02T15:41:27.711497+0000","last_undegraded":"2024-06-02T15:41:27.711497+0000","last_fullsized":"2024-06-02T15:41:27.711497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:50:30.556030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:44.144729+0000","last_change":"2024-06-02T15:38:29.779028+0000","last_active":"2024-06-02T15:40:44.144729+0000","last_peered":"2024-06-02T15:40:44.144729+0000","last_clean":"2024-06-02T15:40:44.144729+0000","last_became_active":"2024-06-02T15:38:29.778575+0000","last_became_peered":"2024-06-02T15:38:29.778575+0000","last_unstale":"2024-06-02T15:40:44.144729+0000","last_undegraded":"2024-06-02T15:40:44.144729+0000","last_fullsized":"2024-06-02T15:40:44.144729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:20:36.424730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.849271+0000","last_change":"2024-06-02T15:38:12.392699+0000","last_active":"2024-06-02T15:40:30.849271+0000","last_peered":"2024-06-02T15:40:30.849271+0000","last_clean":"2024-06-02T15:40:30.849271+0000","last_became_active":"2024-06-02T15:38:12.392321+0000","last_became_peered":"2024-06-02T15:38:12.392321+0000","last_unstale":"2024-06-02T15:40:30.849271+0000","last_undegraded":"2024-06-02T15:40:30.849271+0000","last_fullsized":"2024-06-02T15:40:30.849271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:19:30.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.692818+0000","last_change":"2024-06-02T15:38:16.478921+0000","last_active":"2024-06-02T15:41:27.692818+0000","last_peered":"2024-06-02T15:41:27.692818+0000","last_clean":"2024-06-02T15:41:27.692818+0000","last_became_active":"2024-06-02T15:38:16.478443+0000","last_became_peered":"2024-06-02T15:38:16.478443+0000","last_unstale":"2024-06-02T15:41:27.692818+0000","last_undegraded":"2024-06-02T15:41:27.692818+0000","last_fullsized":"2024-06-02T15:41:27.692818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:08:21.684598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:41.383274+0000","last_change":"2024-06-02T15:38:29.783894+0000","last_active":"2024-06-02T15:40:41.383274+0000","last_peered":"2024-06-02T15:40:41.383274+0000","last_clean":"2024-06-02T15:40:41.383274+0000","last_became_active":"2024-06-02T15:38:29.783440+0000","last_became_peered":"2024-06-02T15:38:29.783440+0000","last_unstale":"2024-06-02T15:40:41.383274+0000","last_undegraded":"2024-06-02T15:40:41.383274+0000","last_fullsized":"2024-06-02T15:40:41.383274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:21:04.280168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:30.645527+0000","last_change":"2024-06-02T15:38:12.517394+0000","last_active":"2024-06-02T15:40:30.645527+0000","last_peered":"2024-06-02T15:40:30.645527+0000","last_clean":"2024-06-02T15:40:30.645527+0000","last_became_active":"2024-06-02T15:38:12.516986+0000","last_became_peered":"2024-06-02T15:38:12.516986+0000","last_unstale":"2024-06-02T15:40:30.645527+0000","last_undegraded":"2024-06-02T15:40:30.645527+0000","last_fullsized":"2024-06-02T15:40:30.645527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:49:53.571142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.694463+0000","last_change":"2024-06-02T15:38:16.463765+0000","last_active":"2024-06-02T15:41:27.694463+0000","last_peered":"2024-06-02T15:41:27.694463+0000","last_clean":"2024-06-02T15:41:27.694463+0000","last_became_active":"2024-06-02T15:38:16.463449+0000","last_became_peered":"2024-06-02T15:38:16.463449+0000","last_unstale":"2024-06-02T15:41:27.694463+0000","last_undegraded":"2024-06-02T15:41:27.694463+0000","last_fullsized":"2024-06-02T15:41:27.694463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:58:21.780878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:46.930876+0000","last_change":"2024-06-02T15:40:00.034227+0000","last_active":"2024-06-02T15:40:46.930876+0000","last_peered":"2024-06-02T15:40:46.930876+0000","last_clean":"2024-06-02T15:40:46.930876+0000","last_became_active":"2024-06-02T15:40:00.033772+0000","last_became_peered":"2024-06-02T15:40:00.033772+0000","last_unstale":"2024-06-02T15:40:46.930876+0000","last_undegraded":"2024-06-02T15:40:46.930876+0000","last_fullsized":"2024-06-02T15:40:46.930876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T16:12:41.138397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:28.928032+0000","last_change":"2024-06-02T15:38:12.405009+0000","last_active":"2024-06-02T15:40:28.928032+0000","last_peered":"2024-06-02T15:40:28.928032+0000","last_clean":"2024-06-02T15:40:28.928032+0000","last_became_active":"2024-06-02T15:38:12.404553+0000","last_became_peered":"2024-06-02T15:38:12.404553+0000","last_unstale":"2024-06-02T15:40:28.928032+0000","last_undegraded":"2024-06-02T15:40:28.928032+0000","last_fullsized":"2024-06-02T15:40:28.928032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:04:52.652613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.705291+0000","last_change":"2024-06-02T15:38:16.470041+0000","last_active":"2024-06-02T15:41:27.705291+0000","last_peered":"2024-06-02T15:41:27.705291+0000","last_clean":"2024-06-02T15:41:27.705291+0000","last_became_active":"2024-06-02T15:38:16.469683+0000","last_became_peered":"2024-06-02T15:38:16.469683+0000","last_unstale":"2024-06-02T15:41:27.705291+0000","last_undegraded":"2024-06-02T15:41:27.705291+0000","last_fullsized":"2024-06-02T15:41:27.705291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:41:03.240581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:38.929210+0000","last_change":"2024-06-02T15:38:29.766662+0000","last_active":"2024-06-02T15:40:38.929210+0000","last_peered":"2024-06-02T15:40:38.929210+0000","last_clean":"2024-06-02T15:40:38.929210+0000","last_became_active":"2024-06-02T15:38:29.766165+0000","last_became_peered":"2024-06-02T15:38:29.766165+0000","last_unstale":"2024-06-02T15:40:38.929210+0000","last_undegraded":"2024-06-02T15:40:38.929210+0000","last_fullsized":"2024-06-02T15:40:38.929210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:28.743091+0000","last_clean_scrub_stamp":"2024-06-02T15:38:28.743091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:57:56.091477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:40:29.462003+0000","last_change":"2024-06-02T15:38:12.406988+0000","last_active":"2024-06-02T15:40:29.462003+0000","last_peered":"2024-06-02T15:40:29.462003+0000","last_clean":"2024-06-02T15:40:29.462003+0000","last_became_active":"2024-06-02T15:38:12.404975+0000","last_became_peered":"2024-06-02T15:38:12.404975+0000","last_unstale":"2024-06-02T15:40:29.462003+0000","last_undegraded":"2024-06-02T15:40:29.462003+0000","last_fullsized":"2024-06-02T15:40:29.462003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:11.372598+0000","last_clean_scrub_stamp":"2024-06-02T15:38:11.372598+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:45:18.544896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.685479+0000","last_change":"2024-06-02T15:38:16.459482+0000","last_active":"2024-06-02T15:41:27.685479+0000","last_peered":"2024-06-02T15:41:27.685479+0000","last_clean":"2024-06-02T15:41:27.685479+0000","last_became_active":"2024-06-02T15:38:16.459150+0000","last_became_peered":"2024-06-02T15:38:16.459150+0000","last_unstale":"2024-06-02T15:41:27.685479+0000","last_undegraded":"2024-06-02T15:41:27.685479+0000","last_fullsized":"2024-06-02T15:41:27.685479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:23:51.144967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.687491+0000","last_change":"2024-06-02T15:38:16.469603+0000","last_active":"2024-06-02T15:41:27.687491+0000","last_peered":"2024-06-02T15:41:27.687491+0000","last_clean":"2024-06-02T15:41:27.687491+0000","last_became_active":"2024-06-02T15:38:16.468927+0000","last_became_peered":"2024-06-02T15:38:16.468927+0000","last_unstale":"2024-06-02T15:41:27.687491+0000","last_undegraded":"2024-06-02T15:41:27.687491+0000","last_fullsized":"2024-06-02T15:41:27.687491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:30:30.516334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686399+0000","last_change":"2024-06-02T15:38:16.464307+0000","last_active":"2024-06-02T15:41:27.686399+0000","last_peered":"2024-06-02T15:41:27.686399+0000","last_clean":"2024-06-02T15:41:27.686399+0000","last_became_active":"2024-06-02T15:38:16.463989+0000","last_became_peered":"2024-06-02T15:38:16.463989+0000","last_unstale":"2024-06-02T15:41:27.686399+0000","last_undegraded":"2024-06-02T15:41:27.686399+0000","last_fullsized":"2024-06-02T15:41:27.686399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T20:27:50.125031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701372+0000","last_change":"2024-06-02T15:38:16.475912+0000","last_active":"2024-06-02T15:41:27.701372+0000","last_peered":"2024-06-02T15:41:27.701372+0000","last_clean":"2024-06-02T15:41:27.701372+0000","last_became_active":"2024-06-02T15:38:16.475595+0000","last_became_peered":"2024-06-02T15:38:16.475595+0000","last_unstale":"2024-06-02T15:41:27.701372+0000","last_undegraded":"2024-06-02T15:41:27.701372+0000","last_fullsized":"2024-06-02T15:41:27.701372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T17:44:35.801446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.701977+0000","last_change":"2024-06-02T15:38:16.485624+0000","last_active":"2024-06-02T15:41:27.701977+0000","last_peered":"2024-06-02T15:41:27.701977+0000","last_clean":"2024-06-02T15:41:27.701977+0000","last_became_active":"2024-06-02T15:38:16.485236+0000","last_became_peered":"2024-06-02T15:38:16.485236+0000","last_unstale":"2024-06-02T15:41:27.701977+0000","last_undegraded":"2024-06-02T15:41:27.701977+0000","last_fullsized":"2024-06-02T15:41:27.701977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T22:50:36.124619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.704468+0000","last_change":"2024-06-02T15:38:16.487679+0000","last_active":"2024-06-02T15:41:27.704468+0000","last_peered":"2024-06-02T15:41:27.704468+0000","last_clean":"2024-06-02T15:41:27.704468+0000","last_became_active":"2024-06-02T15:38:16.486876+0000","last_became_peered":"2024-06-02T15:38:16.486876+0000","last_unstale":"2024-06-02T15:41:27.704468+0000","last_undegraded":"2024-06-02T15:41:27.704468+0000","last_fullsized":"2024-06-02T15:41:27.704468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:19:17.096725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.686919+0000","last_change":"2024-06-02T15:38:16.469264+0000","last_active":"2024-06-02T15:41:27.686919+0000","last_peered":"2024-06-02T15:41:27.686919+0000","last_clean":"2024-06-02T15:41:27.686919+0000","last_became_active":"2024-06-02T15:38:16.468943+0000","last_became_peered":"2024-06-02T15:38:16.468943+0000","last_unstale":"2024-06-02T15:41:27.686919+0000","last_undegraded":"2024-06-02T15:41:27.686919+0000","last_fullsized":"2024-06-02T15:41:27.686919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T19:18:03.802674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.691250+0000","last_change":"2024-06-02T15:38:16.463817+0000","last_active":"2024-06-02T15:41:27.691250+0000","last_peered":"2024-06-02T15:41:27.691250+0000","last_clean":"2024-06-02T15:41:27.691250+0000","last_became_active":"2024-06-02T15:38:16.463444+0000","last_became_peered":"2024-06-02T15:38:16.463444+0000","last_unstale":"2024-06-02T15:41:27.691250+0000","last_undegraded":"2024-06-02T15:41:27.691250+0000","last_fullsized":"2024-06-02T15:41:27.691250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:58:30.684635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.702170+0000","last_change":"2024-06-02T15:38:16.472038+0000","last_active":"2024-06-02T15:41:27.702170+0000","last_peered":"2024-06-02T15:41:27.702170+0000","last_clean":"2024-06-02T15:41:27.702170+0000","last_became_active":"2024-06-02T15:38:16.471120+0000","last_became_peered":"2024-06-02T15:38:16.471120+0000","last_unstale":"2024-06-02T15:41:27.702170+0000","last_undegraded":"2024-06-02T15:41:27.702170+0000","last_fullsized":"2024-06-02T15:41:27.702170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:03:52.136582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711806+0000","last_change":"2024-06-02T15:38:16.480862+0000","last_active":"2024-06-02T15:41:27.711806+0000","last_peered":"2024-06-02T15:41:27.711806+0000","last_clean":"2024-06-02T15:41:27.711806+0000","last_became_active":"2024-06-02T15:38:16.480495+0000","last_became_peered":"2024-06-02T15:38:16.480495+0000","last_unstale":"2024-06-02T15:41:27.711806+0000","last_undegraded":"2024-06-02T15:41:27.711806+0000","last_fullsized":"2024-06-02T15:41:27.711806+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:34:56.869380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.706949+0000","last_change":"2024-06-02T15:38:16.470002+0000","last_active":"2024-06-02T15:41:27.706949+0000","last_peered":"2024-06-02T15:41:27.706949+0000","last_clean":"2024-06-02T15:41:27.706949+0000","last_became_active":"2024-06-02T15:38:16.469631+0000","last_became_peered":"2024-06-02T15:38:16.469631+0000","last_unstale":"2024-06-02T15:41:27.706949+0000","last_undegraded":"2024-06-02T15:41:27.706949+0000","last_fullsized":"2024-06-02T15:41:27.706949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:46:57.498237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.711917+0000","last_change":"2024-06-02T15:38:16.475497+0000","last_active":"2024-06-02T15:41:27.711917+0000","last_peered":"2024-06-02T15:41:27.711917+0000","last_clean":"2024-06-02T15:41:27.711917+0000","last_became_active":"2024-06-02T15:38:16.475054+0000","last_became_peered":"2024-06-02T15:38:16.475054+0000","last_unstale":"2024-06-02T15:41:27.711917+0000","last_undegraded":"2024-06-02T15:41:27.711917+0000","last_fullsized":"2024-06-02T15:41:27.711917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T15:41:48.851319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.703925+0000","last_change":"2024-06-02T15:38:16.480315+0000","last_active":"2024-06-02T15:41:27.703925+0000","last_peered":"2024-06-02T15:41:27.703925+0000","last_clean":"2024-06-02T15:41:27.703925+0000","last_became_active":"2024-06-02T15:38:16.478309+0000","last_became_peered":"2024-06-02T15:38:16.478309+0000","last_unstale":"2024-06-02T15:41:27.703925+0000","last_undegraded":"2024-06-02T15:41:27.703925+0000","last_fullsized":"2024-06-02T15:41:27.703925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T21:52:02.719575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T15:41:27.698726+0000","last_change":"2024-06-02T15:38:59.184466+0000","last_active":"2024-06-02T15:41:27.698726+0000","last_peered":"2024-06-02T15:41:27.698726+0000","last_clean":"2024-06-02T15:41:27.698726+0000","last_became_active":"2024-06-02T15:38:59.184006+0000","last_became_peered":"2024-06-02T15:38:59.184006+0000","last_unstale":"2024-06-02T15:41:27.698726+0000","last_undegraded":"2024-06-02T15:41:27.698726+0000","last_fullsized":"2024-06-02T15:41:27.698726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T15:38:15.413792+0000","last_clean_scrub_stamp":"2024-06-02T15:38:15.413792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T18:03:47.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74662,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1381,"num_read_kb":772,"num_write":185,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":2194,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":223986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874324,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712288,"statfs":{"total":95995035648,"available":95961382912,"internally_reserved":0,"allocated":1728512,"data_stored":1175948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037850,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234081,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":522412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430309,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1114316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593835,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":513572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":583470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738509,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":514960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":515364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149820,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":513766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953591,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":588742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790065,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1646592,"data_stored":1105858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T15:43:24.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:43:24.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:24 smithi090 ceph-mon[31595]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:24.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:43:24.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:43:24.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:43:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:24 smithi120 ceph-mon[36451]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:43:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:43:24.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:43:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:24 smithi135 ceph-mon[36839]: from='client.25187 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:43:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:43:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:43:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:25 smithi090 ceph-mon[31595]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:25 smithi090 ceph-mon[31595]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:25 smithi090 ceph-mon[31595]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:25 smithi120 ceph-mon[36451]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:25 smithi120 ceph-mon[36451]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:25 smithi120 ceph-mon[36451]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:25 smithi135 ceph-mon[36839]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:25 smithi135 ceph-mon[36839]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:25 smithi135 ceph-mon[36839]: from='client.25199 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 -- 172.21.15.90:0/330130636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 msgr2=0x7fd9dc10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/330130636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc10e7d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009920 tx=0x7fd9c4009c30 comp rx=0 tx=0).stop 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 -- 172.21.15.90:0/330130636 shutdown_connections 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/330130636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 0x7fd9dc116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/330130636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/330130636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc10e7d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 -- 172.21.15.90:0/330130636 >> 172.21.15.90:0/330130636 conn(0x7fd9dc076d10 msgr2=0x7fd9dc108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 -- 172.21.15.90:0/330130636 shutdown_connections 2024-06-02T15:43:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.689+0000 7fd9e1a00700 1 -- 172.21.15.90:0/330130636 wait complete. 2024-06-02T15:43:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 Processor -- start 2024-06-02T15:43:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 -- start start 2024-06-02T15:43:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 0x7fd9dc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc119d80 con 0x7fd9dc109dd0 2024-06-02T15:43:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc119c00 con 0x7fd9dc10eeb0 2024-06-02T15:43:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.691+0000 7fd9e1a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc119f00 con 0x7fd9dc10a780 2024-06-02T15:43:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34728/0 (socket says 172.21.15.90:34728) 2024-06-02T15:43:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 -- 172.21.15.90:0/2297851727 learned_addr learned my addr 172.21.15.90:0/2297851727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9d2000700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9db400700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 0x7fd9dc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 -- 172.21.15.90:0/2297851727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 msgr2=0x7fd9dc1a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc1a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 -- 172.21.15.90:0/2297851727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 msgr2=0x7fd9dc1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 0x7fd9dc1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.692+0000 7fd9daa00700 1 -- 172.21.15.90:0/2297851727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1a5330 con 0x7fd9dc109dd0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9daa00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9c0 tx=0x7fd9cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc004a30 con 0x7fd9dc109dd0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9cc00cc70 con 0x7fd9dc109dd0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc1a5620 con 0x7fd9dc109dd0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc005490 con 0x7fd9dc109dd0 2024-06-02T15:43:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.693+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1acea0 con 0x7fd9dc109dd0 2024-06-02T15:43:26.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.694+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc06a230 con 0x7fd9dc109dd0 2024-06-02T15:43:26.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.696+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd9cc004bd0 con 0x7fd9dc109dd0 2024-06-02T15:43:26.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.698+0000 7fd9d3e00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 0x7fd9c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:26.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.698+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fd9cc0e26f0 con 0x7fd9dc109dd0 2024-06-02T15:43:26.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.698+0000 7fd9d2000700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 0x7fd9c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:26.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.699+0000 7fd9d2000700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 0x7fd9c00b4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40098f0 tx=0x7fd9c4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:26.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.703+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9cc0aa000 con 0x7fd9dc109dd0 2024-06-02T15:43:26.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.983+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9dc02e560 con 0x7fd9dc109dd0 2024-06-02T15:43:26.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.985+0000 7fd9d3e00700 1 -- 172.21.15.90:0/2297851727 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7fd9cc0a9900 con 0x7fd9dc109dd0 2024-06-02T15:43:26.983 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:26.983 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:43:15.667079+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:43:26.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 msgr2=0x7fd9c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:26.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 0x7fd9c00b4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40098f0 tx=0x7fd9c4019040 comp rx=0 tx=0).stop 2024-06-02T15:43:26.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 msgr2=0x7fd9dc1a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:26.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00e9c0 tx=0x7fd9cc00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 shutdown_connections 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd9c00b2880 0x7fd9c00b4d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9dc10eeb0 0x7fd9dc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9dc109dd0 0x7fd9dc1a0370 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 --2- 172.21.15.90:0/2297851727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9dc10a780 0x7fd9dc1a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 >> 172.21.15.90:0/2297851727 conn(0x7fd9dc076d10 msgr2=0x7fd9dc078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:26.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 shutdown_connections 2024-06-02T15:43:26.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:26.989+0000 7fd9e1a00700 1 -- 172.21.15.90:0/2297851727 wait complete. 2024-06-02T15:43:26.988 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-06-02T15:43:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:27 smithi120 ceph-mon[36451]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:27 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2297851727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:27 smithi135 ceph-mon[36839]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:27 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2297851727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:27 smithi090 ceph-mon[31595]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:27 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2297851727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:43:27.843 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:43:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:29.468 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:29 smithi090 ceph-mon[31595]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:29 smithi120 ceph-mon[36451]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:29 smithi135 ceph-mon[36839]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 -- 172.21.15.90:0/3458337262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd081096a0 msgr2=0x7efd08109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 --2- 172.21.15.90:0/3458337262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd081096a0 0x7efd08109a80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009990 tx=0x7efcf8009ca0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 -- 172.21.15.90:0/3458337262 shutdown_connections 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 --2- 172.21.15.90:0/3458337262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd08110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 --2- 172.21.15.90:0/3458337262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd081096a0 0x7efd08109a80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 --2- 172.21.15.90:0/3458337262 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd08103590 0x7efd08103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 -- 172.21.15.90:0/3458337262 >> 172.21.15.90:0/3458337262 conn(0x7efd0807a6f0 msgr2=0x7efd0807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:30.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 -- 172.21.15.90:0/3458337262 shutdown_connections 2024-06-02T15:43:30.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.845+0000 7efd0dc00700 1 -- 172.21.15.90:0/3458337262 wait complete. 2024-06-02T15:43:30.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 Processor -- start 2024-06-02T15:43:30.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 -- start start 2024-06-02T15:43:30.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:30.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd0819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 0x7efd08199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd08113710 con 0x7efd08103590 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd08113590 con 0x7efd081096a0 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.847+0000 7efd0dc00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd08113890 con 0x7efd08103f50 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34744/0 (socket says 172.21.15.90:34744) 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 -- 172.21.15.90:0/1691428773 learned_addr learned my addr 172.21.15.90:0/1691428773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06000700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd0819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd07400700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 0x7efd08199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 -- 172.21.15.90:0/1691428773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 msgr2=0x7efd0819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd0819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 -- 172.21.15.90:0/1691428773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 msgr2=0x7efd08199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 0x7efd08199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06a00700 1 -- 172.21.15.90:0/1691428773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0819a0b0 con 0x7efd08103590 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd06000700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd0819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.848+0000 7efd07400700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 0x7efd08199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efd06a00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcf80057f0 tx=0x7efcf800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8026070 con 0x7efd08103590 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf8021410 con 0x7efd08103590 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8017650 con 0x7efd08103590 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0819a340 con 0x7efd08103590 2024-06-02T15:43:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.849+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a6660 con 0x7efd08103590 2024-06-02T15:43:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.850+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0806a230 con 0x7efd08103590 2024-06-02T15:43:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.852+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efcf8029070 con 0x7efd08103590 2024-06-02T15:43:30.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.855+0000 7efcff400700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 0x7efcec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:30.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.855+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efcf80e34b0 con 0x7efd08103590 2024-06-02T15:43:30.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.855+0000 7efd06000700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 0x7efcec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:30.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.856+0000 7efd06000700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 0x7efcec0b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd0807ac60 tx=0x7efcf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:30.858+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efcf8030080 con 0x7efd08103590 2024-06-02T15:43:31.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.095+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd0802e560 con 0x7efd08103590 2024-06-02T15:43:31.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.097+0000 7efcff400700 1 -- 172.21.15.90:0/1691428773 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24751 (secure 0 0 0) 0x7efcf8013070 con 0x7efd08103590 2024-06-02T15:43:31.095 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:43:31.095 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:40:00.011486+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:43:31.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 msgr2=0x7efcec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 0x7efcec0b4d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd0807ac60 tx=0x7efcf4009260 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 msgr2=0x7efd0819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcf80057f0 tx=0x7efcf800bfd0 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 shutdown_connections 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efcec0b2880 0x7efcec0b4d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd081096a0 0x7efd08199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd08103590 0x7efd0819f790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 --2- 172.21.15.90:0/1691428773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd08103f50 0x7efd0819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 >> 172.21.15.90:0/1691428773 conn(0x7efd0807a6f0 msgr2=0x7efd08102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.100+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 shutdown_connections 2024-06-02T15:43:31.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:31.101+0000 7efd0dc00700 1 -- 172.21.15.90:0/1691428773 wait complete. 2024-06-02T15:43:31.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:31 smithi090 ceph-mon[31595]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-02T15:43:31.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:31 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1691428773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:31 smithi120 ceph-mon[36451]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-02T15:43:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:31 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1691428773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:31 smithi135 ceph-mon[36839]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-06-02T15:43:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:31 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1691428773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:43:32.207 DEBUG:tasks.fwd_scrub:Yielding 2024-06-02T15:43:32.207 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-02T15:43:32.217 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:43:32.217 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-02T15:43:32.218 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-02T15:43:32.218 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:43:32.218 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:43:32.224 INFO:teuthology.run_tasks:Running task workunit... 2024-06-02T15:43:32.237 INFO:tasks.workunit:Pulling workunits from ref 7aec00165b4b073a25396fd0279151d2b2fa23fe 2024-06-02T15:43:32.237 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-02T15:43:32.237 INFO:tasks.workunit:timeout=3h 2024-06-02T15:43:32.237 INFO:tasks.workunit:cleanup=True 2024-06-02T15:43:32.238 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-06-02 15:39:56.086799651 +0000 2024-06-02T15:43:32.283 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-06-02 15:39:56.086799651 +0000 2024-06-02T15:43:32.284 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-06-02 15:43:14.632326480 +0000 2024-06-02T15:43:32.284 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-06-02 15:39:56.086799651 +0000 2024-06-02T15:43:32.284 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-02T15:43:32.284 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-02T15:43:32.403 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7aec00165b4b073a25396fd0279151d2b2fa23fe 2024-06-02T15:43:32.479 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-02T15:43:32.653 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:33.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:33 smithi120 ceph-mon[36451]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:43:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:33 smithi135 ceph-mon[36839]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:43:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:33 smithi090 ceph-mon[31595]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:43:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:35 smithi090 ceph-mon[31595]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 -- 172.21.15.90:0/3664583927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406b220 msgr2=0x7fdd7406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 --2- 172.21.15.90:0/3664583927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406b220 0x7fdd7406b6a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009990 tx=0x7fdd60009ca0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 -- 172.21.15.90:0/3664583927 shutdown_connections 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 --2- 172.21.15.90:0/3664583927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd7406bbe0 0x7fdd741103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 --2- 172.21.15.90:0/3664583927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd741095b0 0x7fdd74109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 --2- 172.21.15.90:0/3664583927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406b220 0x7fdd7406b6a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 -- 172.21.15.90:0/3664583927 >> 172.21.15.90:0/3664583927 conn(0x7fdd740793f0 msgr2=0x7fdd74079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.287+0000 7fdd7b000700 1 -- 172.21.15.90:0/3664583927 shutdown_connections 2024-06-02T15:43:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.288+0000 7fdd7b000700 1 -- 172.21.15.90:0/3664583927 wait complete. 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.288+0000 7fdd7b000700 1 Processor -- start 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 -- start start 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd7406b220 0x7fdd7419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd741095b0 0x7fdd74199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741136e0 con 0x7fdd7406bbe0 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74113560 con 0x7fdd741095b0 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd7b000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd74113860 con 0x7fdd7406b220 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57180/0 (socket says 172.21.15.90:57180) 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd73400700 1 -- 172.21.15.90:0/1154262362 learned_addr learned my addr 172.21.15.90:0/1154262362 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd79000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd741095b0 0x7fdd74199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.289+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd7406b220 0x7fdd7419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 -- 172.21.15.90:0/1154262362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd7406b220 msgr2=0x7fdd7419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd7406b220 0x7fdd7419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 -- 172.21.15.90:0/1154262362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd741095b0 msgr2=0x7fdd74199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd741095b0 0x7fdd74199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 -- 172.21.15.90:0/1154262362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7419a200 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd73400700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd60000c00 tx=0x7fdd60009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd70c00700 1 -- 172.21.15.90:0/1154262362 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60027070 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd70c00700 1 -- 172.21.15.90:0/1154262362 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd60022470 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.290+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7419a490 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.291+0000 7fdd70c00700 1 -- 172.21.15.90:0/1154262362 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60017650 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.291+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd741a67b0 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.295+0000 7fdd70c00700 1 -- 172.21.15.90:0/1154262362 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdd6002a070 con 0x7fdd7406bbe0 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.295+0000 7fdd8595bb80 1 Processor -- start 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.295+0000 7fdd8595bb80 1 -- start start 2024-06-02T15:43:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd5baa0 0x555a5bd5be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd565d0 0x555a5bd56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555a5bcafec0 con 0x555a5bd5baa0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a5bc9fa60 con 0x555a5bd5c3c0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a5bca3aa0 con 0x555a5bd565d0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35214/0 (socket says 172.21.15.90:35214) 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd73e00700 1 -- 172.21.15.90:0/2921129198 learned_addr learned my addr 172.21.15.90:0/2921129198 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd73400700 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd565d0 0x555a5bd56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd79000700 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd5baa0 0x555a5bd5be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.296+0000 7fdd70c00700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 0x7fdd580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd70c00700 1 -- 172.21.15.90:0/1154262362 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdd600e3340 con 0x7fdd7406bbe0 2024-06-02T15:43:35.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 -- 172.21.15.90:0/2921129198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd565d0 msgr2=0x555a5bd56a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd565d0 0x555a5bd56a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 -- 172.21.15.90:0/2921129198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd5baa0 msgr2=0x555a5bd5be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd5baa0 0x555a5bd5be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 -- 172.21.15.90:0/2921129198 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a5bc684a0 con 0x555a5bd5c3c0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd79000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 0x7fdd580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.297+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009cb0 tx=0x7fdd64011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd21d8417d86ca1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.298+0000 7fdd79000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 0x7fdd580b4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009550 tx=0x7fdd6c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.300+0000 7fdd69600700 1 -- 172.21.15.90:0/2921129198 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64011d80 con 0x555a5bd5c3c0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.300+0000 7fdd69600700 1 -- 172.21.15.90:0/2921129198 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd64004ce0 con 0x555a5bd5c3c0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.300+0000 7fdd69600700 1 -- 172.21.15.90:0/2921129198 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64019690 con 0x555a5bd5c3c0 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2921129198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 msgr2=0x555a5bd55fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009cb0 tx=0x7fdd64011440 comp rx=0 tx=0).stop 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2921129198 shutdown_connections 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd565d0 0x555a5bd56a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd5baa0 0x555a5bd5be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2921129198 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5c3c0 0x555a5bd55fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.301+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2921129198 >> 172.21.15.90:0/2921129198 conn(0x555a5bc7fc60 msgr2=0x555a5bd4c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.302+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2921129198 shutdown_connections 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.302+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2921129198 wait complete. 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.302+0000 7fdd8595bb80 1 -- start start 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.302+0000 7fdd8595bb80 1 Processor -- start 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd61ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bd62430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5c3c0 0x555a5bd5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555a5bba6020 con 0x555a5bd565d0 2024-06-02T15:43:35.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a5bc407e0 con 0x555a5bd5baa0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a5bd62ee0 con 0x555a5bd5c3c0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd8595bb80 10 client.0 _ll_register_callbacks cb 0x555a5bc49cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd61ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd61ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57206/0 (socket says 172.21.15.90:57206) 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd73400700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bd62430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd79000700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5c3c0 0x555a5bd5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.303+0000 7fdd73e00700 1 -- 172.21.15.90:0/1585753164 learned_addr learned my addr 172.21.15.90:0/1585753164 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 -- 172.21.15.90:0/1585753164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5c3c0 msgr2=0x555a5bd5f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5c3c0 0x555a5bd5f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 -- 172.21.15.90:0/1585753164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 msgr2=0x555a5bd61ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd61ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 -- 172.21.15.90:0/1585753164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a5bbef950 con 0x555a5bd5baa0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd73400700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bd62430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd600a6660 tx=0x7fdd60022c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf75b5a334286af server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd5e000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd600a9f30 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdd600b3120 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.304+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd64004ce0 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd600a9d50 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 10 client.25223 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a5bc700e0 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd64004ce0 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdd600a9840 con 0x555a5bd5baa0 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 10 client.25223 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 20 client.25223 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd73e80 0x555a5bd762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 4 client.25223 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.305+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a5bcb1e10 con 0x555a5bd73e80 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.306+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd73e80 0x555a5bd762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.306+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd73e80 0x555a5bd762e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e96967f26b75cc server_cookie=d1be9573279473e0 in_seq=0 out_seq=0 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.306+0000 7fdd5e000700 10 client.25223 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.307+0000 7fdd5e000700 1 -- 172.21.15.90:0/1585753164 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555a5bcb1e10 con 0x555a5bd73e80 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.307+0000 7fdd5e000700 10 client.25223 handle_command_reply: tid=0 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.307+0000 7fdd8595bb80 1 client.25223 shutdown 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 msgr2=0x555a5bd62430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bd62430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdd600a6660 tx=0x7fdd60022c20 comp rx=0 tx=0).stop 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 shutdown_connections 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd73e80 0x555a5bd762e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5c3c0 0x555a5bd5f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd61ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd5e000700 0 client.25223 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1585753164 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bd62430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 >> 172.21.15.90:0/1585753164 conn(0x555a5bc7fc60 msgr2=0x555a5bd4bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 shutdown_connections 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1585753164 wait complete. 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.308+0000 7fdd8595bb80 20 client.25223 trim_cache size 0 max 16384 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 Processor -- start 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 -- start start 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd5d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 0x555a5bd4c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741af840 con 0x555a5bd565d0 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741ad0d0 con 0x555a5bc7fc60 2024-06-02T15:43:35.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.331+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aaf20 con 0x555a5bd5baa0 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35244/0 (socket says 172.21.15.90:35244) 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 -- 172.21.15.90:0/1825275523 learned_addr learned my addr 172.21.15.90:0/1825275523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd79000700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 0x555a5bd4c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73400700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd5d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 -- 172.21.15.90:0/1825275523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 msgr2=0x555a5bd4c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 0x555a5bd4c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 -- 172.21.15.90:0/1825275523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 msgr2=0x555a5bd5d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd5d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd73e00700 1 -- 172.21.15.90:0/1825275523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a5bd5c1f0 con 0x555a5bc7fc60 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.332+0000 7fdd79000700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 0x555a5bd4c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.333+0000 7fdd73e00700 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009cb0 tx=0x7fdd640051e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860a259af02f84d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.333+0000 7fdd68c00700 1 -- 172.21.15.90:0/1825275523 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030070 con 0x555a5bc7fc60 2024-06-02T15:43:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.333+0000 7fdd68c00700 1 -- 172.21.15.90:0/1825275523 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd64005c20 con 0x555a5bc7fc60 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.333+0000 7fdd68c00700 1 -- 172.21.15.90:0/1825275523 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6402a550 con 0x555a5bc7fc60 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1825275523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 msgr2=0x555a5bd5e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009cb0 tx=0x7fdd640051e0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1825275523 shutdown_connections 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bd5baa0 0x555a5bd4c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bc7fc60 0x555a5bd5e690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/1825275523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd5d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1825275523 >> 172.21.15.90:0/1825275523 conn(0x555a5bd56d10 msgr2=0x555a5bd5a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1825275523 shutdown_connections 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.334+0000 7fdd8595bb80 1 -- 172.21.15.90:0/1825275523 wait complete. 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 -- start start 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 Processor -- start 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 0x555a5bd6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bdddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741ab590 con 0x555a5bd565d0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741aca00 con 0x555a5bd5baa0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd741abc60 con 0x555a5bc7fc60 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.335+0000 7fdd8595bb80 10 client.0 _ll_register_callbacks cb 0x555a5bc49cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57228/0 (socket says 172.21.15.90:57228) 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73400700 1 -- 172.21.15.90:0/2828328205 learned_addr learned my addr 172.21.15.90:0/2828328205 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 0x555a5bd6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd79000700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bdddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 -- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 msgr2=0x555a5bdddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bdddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 -- 172.21.15.90:0/2828328205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 msgr2=0x555a5bd6f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73e00700 1 -- 172.21.15.90:0/2828328205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a5bdde040 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.336+0000 7fdd73400700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd73e00700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 0x555a5bd6f090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd64011790 tx=0x7fdd64009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f106c29c218dde76 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd5e000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64030070 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a5bcab5f0 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdd64011220 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd64003680 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd8595bb80 10 client.25201 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a5bddf380 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.337+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555a5bcab5f0 con 0x555a5bc7fc60 2024-06-02T15:43:35.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fdd6402b040 con 0x555a5bc7fc60 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 10 client.25201 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 10 client.25201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 20 client.25201 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd5c3c0 0x555a5bde17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 4 client.25201 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a5bb222d0 con 0x555a5bd5c3c0 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.338+0000 7fdd73400700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd5c3c0 0x555a5bde17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.339+0000 7fdd73400700 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd5c3c0 0x555a5bde17e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d893f7f652ac3f66 server_cookie=ce3d3c81f3ad679e in_seq=0 out_seq=0 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.339+0000 7fdd5e000700 10 client.25201 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd5e000700 1 -- 172.21.15.90:0/2828328205 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a5bb222d0 con 0x555a5bd5c3c0 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd5e000700 10 client.25201 handle_command_reply: tid=0 2024-06-02T15:43:35.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 client.25201 shutdown 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "0c2a690b-68e9-47bc-97da-fdf78a80cf7e", 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:43:35.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 msgr2=0x555a5bd6f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 0x555a5bd6f090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd64011790 tx=0x7fdd64009740 comp rx=0 tx=0).stop 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 shutdown_connections 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a5bd5c3c0 0x555a5bde17e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a5bd5baa0 0x555a5bdddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a5bc7fc60 0x555a5bd6f090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 --2- 172.21.15.90:0/2828328205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a5bd565d0 0x555a5bd6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd5e000700 0 client.25201 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 >> 172.21.15.90:0/2828328205 conn(0x555a5bd56d10 msgr2=0x555a5bd5a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 shutdown_connections 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.340+0000 7fdd8595bb80 1 -- 172.21.15.90:0/2828328205 wait complete. 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.341+0000 7fdd8595bb80 20 client.25201 trim_cache size 0 max 16384 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.341+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 msgr2=0x7fdd580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.341+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 0x7fdd580b4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009550 tx=0x7fdd6c00b040 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.341+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 msgr2=0x7fdd7419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.341+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd60000c00 tx=0x7fdd60009580 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 shutdown_connections 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdd580b2880 0x7fdd580b4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd741095b0 0x7fdd74199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd7406b220 0x7fdd7419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 --2- 172.21.15.90:0/1154262362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd7406bbe0 0x7fdd7419fe20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 >> 172.21.15.90:0/1154262362 conn(0x7fdd740793f0 msgr2=0x7fdd74101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 shutdown_connections 2024-06-02T15:43:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:35.342+0000 7fdd7b000700 1 -- 172.21.15.90:0/1154262362 wait complete. 2024-06-02T15:43:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:35 smithi120 ceph-mon[36451]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-06-02T15:43:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:35 smithi135 ceph-mon[36839]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-06-02T15:43:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:36 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:43:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:36 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:36 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:43:36.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:36 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:36 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:43:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:36 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:36 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:43:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:36 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:36 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:43:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:36 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:36 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:43:36.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:36 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:43:36.810 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:43:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:43:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:37 smithi135 ceph-mon[36839]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:43:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:37 smithi090 ceph-mon[31595]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:43:37.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:37 smithi120 ceph-mon[36451]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:43:39.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:39 smithi090 ceph-mon[31595]: pgmap v490: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:39 smithi135 ceph-mon[36839]: pgmap v490: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:39 smithi120 ceph-mon[36451]: pgmap v490: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 -- 172.21.15.90:0/2868788859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81037f0 msgr2=0x7f83e8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 --2- 172.21.15.90:0/2868788859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81037f0 0x7f83e8103bd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0099c0 tx=0x7f83dc009cd0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 -- 172.21.15.90:0/2868788859 shutdown_connections 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 --2- 172.21.15.90:0/2868788859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81088f0 0x7f83e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 --2- 172.21.15.90:0/2868788859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81037f0 0x7f83e8103bd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 --2- 172.21.15.90:0/2868788859 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81041a0 0x7f83e81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 -- 172.21.15.90:0/2868788859 >> 172.21.15.90:0/2868788859 conn(0x7f83e807a6a0 msgr2=0x7f83e807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 -- 172.21.15.90:0/2868788859 shutdown_connections 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.053+0000 7f83ef400700 1 -- 172.21.15.90:0/2868788859 wait complete. 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.054+0000 7f83ef400700 1 Processor -- start 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.054+0000 7f83ef400700 1 -- start start 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 0x7f83e8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81088f0 0x7f83e80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8113710 con 0x7f83e81088f0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8113590 con 0x7f83e81037f0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ef400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8113890 con 0x7f83e81041a0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83ed400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81088f0 0x7f83e80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83eca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 0x7f83e8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83eca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35250/0 (socket says 172.21.15.90:35250) 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83eca00700 1 -- 172.21.15.90:0/3585219076 learned_addr learned my addr 172.21.15.90:0/3585219076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.055+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 0x7f83e8075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41896/0 (socket says 172.21.15.90:41896) 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 -- 172.21.15.90:0/3585219076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 msgr2=0x7f83e8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 0x7f83e8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 -- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81088f0 msgr2=0x7f83e80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81088f0 0x7f83e80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 -- 172.21.15.90:0/3585219076 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e81a8150 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83eca00700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006010 tx=0x7f83dc005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83e5600700 1 -- 172.21.15.90:0/3585219076 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc026070 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83e5600700 1 -- 172.21.15.90:0/3585219076 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83dc005680 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e81a83e0 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.056+0000 7f83e5600700 1 -- 172.21.15.90:0/3585219076 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00fb20 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.057+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a8830 con 0x7f83e81037f0 2024-06-02T15:43:40.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.060+0000 7f83e5600700 1 -- 172.21.15.90:0/3585219076 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f83dc029070 con 0x7f83e81037f0 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 Processor -- start 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 -- start start 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196ef9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196ef3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c196e51e40 con 0x55c196ef8eb0 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c196e419e0 con 0x55c196ef3880 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.062+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c196e45a20 con 0x55c196ef97d0 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e5600700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 0x7f83d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83ed400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196ef9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83ed400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196ef9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57254/0 (socket says 172.21.15.90:57254) 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83ed400700 1 -- 172.21.15.90:0/152159598 learned_addr learned my addr 172.21.15.90:0/152159598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:40.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e5600700 1 -- 172.21.15.90:0/3585219076 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f83dc0e2dd0 con 0x7f83e81037f0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83eca00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196ef3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83ed400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 0x7f83d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 -- 172.21.15.90:0/152159598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 msgr2=0x55c196ef3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196ef3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 -- 172.21.15.90:0/152159598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 msgr2=0x55c196ef9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196ef9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83e7e00700 1 -- 172.21.15.90:0/152159598 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c196e0a420 con 0x55c196ef3880 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.063+0000 7f83eca00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196ef3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.064+0000 7f83e7e00700 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83e80772d0 tx=0x7f83d4009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8aa237bd15f97a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.064+0000 7f83da000700 1 -- 172.21.15.90:0/152159598 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011c80 con 0x55c196ef3880 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.064+0000 7f83ed400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 0x7f83d00b4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83e000fb80 tx=0x7f83e0004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.064+0000 7f83da000700 1 -- 172.21.15.90:0/152159598 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d4011e20 con 0x55c196ef3880 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.064+0000 7f83da000700 1 -- 172.21.15.90:0/152159598 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4005280 con 0x55c196ef3880 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 -- 172.21.15.90:0/152159598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 msgr2=0x55c196ef3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83e80772d0 tx=0x7f83d4009f60 comp rx=0 tx=0).stop 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 -- 172.21.15.90:0/152159598 shutdown_connections 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3ce0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196ef9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/152159598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196ef3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 -- 172.21.15.90:0/152159598 >> 172.21.15.90:0/152159598 conn(0x55c196e21be0 msgr2=0x55c196eee010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 -- 172.21.15.90:0/152159598 shutdown_connections 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.065+0000 7f83f9e44b80 1 -- 172.21.15.90:0/152159598 wait complete. 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 -- start start 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 Processor -- start 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196efc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef97d0 0x55c196efcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c196d47fa0 con 0x55c196ef8eb0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c196de2760 con 0x55c196ef97d0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c196f00240 con 0x55c196ef3880 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.066+0000 7f83f9e44b80 10 client.0 _ll_register_callbacks cb 0x55c196debc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41916/0 (socket says 172.21.15.90:41916) 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 -- 172.21.15.90:0/1059530269 learned_addr learned my addr 172.21.15.90:0/1059530269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83ed400700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef97d0 0x55c196efcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196efc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 -- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef97d0 msgr2=0x55c196efcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef97d0 0x55c196efcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 -- 172.21.15.90:0/1059530269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 msgr2=0x55c196efc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196efc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 -- 172.21.15.90:0/1059530269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c196d918d0 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83eca00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0058b0 tx=0x7f83dc0aa5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ffa7c706c17f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.067+0000 7f83cea00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0a9b80 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f83dc0ae0d0 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d4011e20 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83dc0aa770 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83f9e44b80 10 client.25213 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c196e12060 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d4011e20 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f83dc0a5b70 con 0x55c196ef3880 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83f9e44b80 10 client.25213 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:40.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.068+0000 7f83f9e44b80 10 client.25213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.069+0000 7f83f9e44b80 20 client.25213 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.069+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f0cc60 0x55c196f0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.069+0000 7f83f9e44b80 4 client.25213 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.069+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c196e53d90 con 0x55c196f0cc60 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.069+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f0cc60 0x55c196f0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.070+0000 7f83cea00700 10 client.25213 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.070+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f0cc60 0x55c196f0f070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1673d1d9c44f05 server_cookie=f6e9e1da8e31e811 in_seq=0 out_seq=0 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.070+0000 7f83cea00700 1 -- 172.21.15.90:0/1059530269 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c196e53d90 con 0x55c196f0cc60 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.070+0000 7f83cea00700 10 client.25213 handle_command_reply: tid=0 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.070+0000 7f83f9e44b80 1 client.25213 shutdown 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 msgr2=0x55c196eff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0058b0 tx=0x7f83dc0aa5b0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 shutdown_connections 2024-06-02T15:43:40.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f0cc60 0x55c196f0f070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef97d0 0x55c196efcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef3880 0x55c196eff4a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1059530269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196efc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83cea00700 0 client.25213 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 >> 172.21.15.90:0/1059530269 conn(0x55c196e21be0 msgr2=0x55c196ef2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 shutdown_connections 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1059530269 wait complete. 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.071+0000 7f83f9e44b80 20 client.25213 trim_cache size 0 max 16384 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.093+0000 7f83f9e44b80 1 Processor -- start 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.093+0000 7f83f9e44b80 1 -- start start 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196eedf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81b3d80 con 0x55c196ef8eb0 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81b1610 con 0x55c196ef3880 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81af460 con 0x55c196ef97d0 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57280/0 (socket says 172.21.15.90:57280) 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 -- 172.21.15.90:0/3660731012 learned_addr learned my addr 172.21.15.90:0/3660731012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83eca00700 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83ed400700 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196eedf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 -- 172.21.15.90:0/3660731012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 msgr2=0x55c196eedf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196eedf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 -- 172.21.15.90:0/3660731012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 msgr2=0x55c196ef3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.094+0000 7f83e7e00700 1 -- 172.21.15.90:0/3660731012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c196efd0b0 con 0x55c196ef8eb0 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.095+0000 7f83e7e00700 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009bd0 tx=0x7f83d4005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9883e0bd92225743 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.095+0000 7f83d9600700 1 -- 172.21.15.90:0/3660731012 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4011c80 con 0x55c196ef8eb0 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.095+0000 7f83d9600700 1 -- 172.21.15.90:0/3660731012 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d4011e20 con 0x55c196ef8eb0 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.095+0000 7f83d9600700 1 -- 172.21.15.90:0/3660731012 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4004500 con 0x55c196ef8eb0 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- 172.21.15.90:0/3660731012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 msgr2=0x55c196eed9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009bd0 tx=0x7f83d4005ce0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- 172.21.15.90:0/3660731012 shutdown_connections 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196eedf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196ef3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/3660731012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196eed9e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- 172.21.15.90:0/3660731012 >> 172.21.15.90:0/3660731012 conn(0x55c196e21be0 msgr2=0x55c196efb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- 172.21.15.90:0/3660731012 shutdown_connections 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- 172.21.15.90:0/3660731012 wait complete. 2024-06-02T15:43:40.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.096+0000 7f83f9e44b80 1 -- start start 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 Processor -- start 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196f054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196f024a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81afad0 con 0x55c196ef8eb0 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81b0f40 con 0x55c196ef3880 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81b01a0 con 0x55c196ef97d0 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.097+0000 7f83f9e44b80 10 client.0 _ll_register_callbacks cb 0x55c196debc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57292/0 (socket says 172.21.15.90:57292) 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 -- 172.21.15.90:0/1126986707 learned_addr learned my addr 172.21.15.90:0/1126986707 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83eca00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196f054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83ed400700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196f024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 -- 172.21.15.90:0/1126986707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 msgr2=0x55c196f024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196f024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 -- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 msgr2=0x55c196f054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196f054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 -- 172.21.15.90:0/1126986707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c196f02a60 con 0x55c196ef8eb0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83eca00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196f054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83e7e00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83d4004380 tx=0x7f83d40117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee4b56eb701106 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.098+0000 7f83cea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4004050 con 0x55c196ef8eb0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f83d402a040 con 0x55c196ef8eb0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d4011e20 con 0x55c196ef8eb0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83d403f050 con 0x55c196ef8eb0 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83f9e44b80 10 client.15129 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c196f780d0 con 0x55c196ef8eb0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d4011e20 con 0x55c196ef8eb0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.099+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f83d4018070 con 0x55c196ef8eb0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 10 client.15129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 20 client.15129 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f784d0 0x55c196f7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 4 client.15129 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c196e4d570 con 0x55c196f784d0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83eca00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f784d0 0x55c196f7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83eca00700 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f784d0 0x55c196f7a8c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f0e648c57a485d server_cookie=a1b4b0123456ce43 in_seq=0 out_seq=0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.100+0000 7f83cea00700 10 client.15129 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83cea00700 1 -- 172.21.15.90:0/1126986707 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c196e4d570 con 0x55c196f784d0 2024-06-02T15:43:40.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83cea00700 10 client.15129 handle_command_reply: tid=0 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 client.15129 shutdown 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 msgr2=0x55c196f05a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83d4004380 tx=0x7f83d40117a0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 shutdown_connections 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c196f784d0 0x55c196f7a8c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c196ef97d0 0x55c196f024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c196ef3880 0x55c196f054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 --2- 172.21.15.90:0/1126986707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c196ef8eb0 0x55c196f05a10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83cea00700 0 client.15129 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:40.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 >> 172.21.15.90:0/1126986707 conn(0x55c196e21be0 msgr2=0x55c196eed9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "0c2a690b-68e9-47bc-97da-fdf78a80cf7e": { 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "0c2a690b-68e9-47bc-97da-fdf78a80cf7e", 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:43:40.158 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 shutdown_connections 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 1 -- 172.21.15.90:0/1126986707 wait complete. 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.101+0000 7f83f9e44b80 20 client.15129 trim_cache size 0 max 16384 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 msgr2=0x7f83d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 0x7f83d00b4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f83e000fb80 tx=0x7f83e0004b40 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 msgr2=0x7f83e8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f83dc006010 tx=0x7f83dc005280 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 shutdown_connections 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f83d00b2880 0x7f83d00b4d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83e81088f0 0x7f83e80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f83e81037f0 0x7f83e8075840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 --2- 172.21.15.90:0/3585219076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e81041a0 0x7f83e8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:40.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 >> 172.21.15.90:0/3585219076 conn(0x7f83e807a6a0 msgr2=0x7f83e8109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:40.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 shutdown_connections 2024-06-02T15:43:40.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:43:40.102+0000 7f83ef400700 1 -- 172.21.15.90:0/3585219076 wait complete. 2024-06-02T15:43:41.325 INFO:tasks.cephfs.filesystem:scrub status for tag:0c2a690b-68e9-47bc-97da-fdf78a80cf7e - {'path': '/', 'tag': '0c2a690b-68e9-47bc-97da-fdf78a80cf7e', 'options': 'recursive,force'} 2024-06-02T15:43:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:41 smithi090 ceph-mon[31595]: pgmap v491: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:41 smithi120 ceph-mon[36451]: pgmap v491: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:41.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:41 smithi135 ceph-mon[36839]: pgmap v491: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:43 smithi090 ceph-mon[31595]: pgmap v492: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-02T15:43:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:43 smithi120 ceph-mon[36451]: pgmap v492: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-02T15:43:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:43 smithi135 ceph-mon[36839]: pgmap v492: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 5 op/s 2024-06-02T15:43:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 -- 172.21.15.90:0/2114402943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad01066c0 msgr2=0x7ffad010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 --2- 172.21.15.90:0/2114402943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad01066c0 0x7ffad010cf50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009ca0 tx=0x7ffac802a8c0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 -- 172.21.15.90:0/2114402943 shutdown_connections 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 --2- 172.21.15.90:0/2114402943 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad01066c0 0x7ffad010cf50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 --2- 172.21.15.90:0/2114402943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0105d00 0x7ffad0106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 --2- 172.21.15.90:0/2114402943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad0104b10 0x7ffad0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 -- 172.21.15.90:0/2114402943 >> 172.21.15.90:0/2114402943 conn(0x7ffad01000c0 msgr2=0x7ffad01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 -- 172.21.15.90:0/2114402943 shutdown_connections 2024-06-02T15:43:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.474+0000 7ffad6400640 1 -- 172.21.15.90:0/2114402943 wait complete. 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.475+0000 7ffad6400640 1 Processor -- start 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.475+0000 7ffad6400640 1 -- start start 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.475+0000 7ffad6400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.475+0000 7ffad6400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad0105d00 0x7ffad019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad6400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 0x7ffad01a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad6400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0110640 con 0x7ffad0104b10 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad6400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01104c0 con 0x7ffad0105d00 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad6400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01107c0 con 0x7ffad01066c0 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60352/0 (socket says 172.21.15.90:60352) 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 -- 172.21.15.90:0/1468842167 learned_addr learned my addr 172.21.15.90:0/1468842167 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffacfe00640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad0105d00 0x7ffad019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5a00640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 0x7ffad01a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 -- 172.21.15.90:0/1468842167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 msgr2=0x7ffad01a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 0x7ffad01a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 -- 172.21.15.90:0/1468842167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad0105d00 msgr2=0x7ffad019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad0105d00 0x7ffad019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5000640 1 -- 172.21.15.90:0/1468842167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad01a2900 con 0x7ffad0104b10 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.476+0000 7ffad5a00640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 0x7ffad01a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:43:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffad5000640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9c0 tx=0x7ffac400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffacd600640 1 -- 172.21.15.90:0/1468842167 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac400cd80 con 0x7ffad0104b10 2024-06-02T15:43:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffacd600640 1 -- 172.21.15.90:0/1468842167 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac40035f0 con 0x7ffad0104b10 2024-06-02T15:43:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad01a2bf0 con 0x7ffad0104b10 2024-06-02T15:43:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffacd600640 1 -- 172.21.15.90:0/1468842167 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4010690 con 0x7ffad0104b10 2024-06-02T15:43:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.477+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a30f0 con 0x7ffad0104b10 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.480+0000 7ffada855740 1 Processor -- start 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.480+0000 7ffacd600640 1 -- 172.21.15.90:0/1468842167 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffac4004000 con 0x7ffad0104b10 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.480+0000 7ffada855740 1 -- start start 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 0x5624ce0ef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0f4850 0x5624ce0f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624ce05eb90 con 0x5624ce0eecf0 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624ce04e730 con 0x5624ce0ef6e0 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffada855740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ce052770 con 0x5624ce0f4850 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffad5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffad5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 0x5624ce0ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffad5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37398/0 (socket says 172.21.15.90:37398) 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0f4850 0x5624ce0f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.481+0000 7ffad5000640 1 -- 172.21.15.90:0/3021411347 learned_addr learned my addr 172.21.15.90:0/3021411347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 -- 172.21.15.90:0/3021411347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0f4850 msgr2=0x5624ce0f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0f4850 0x5624ce0f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 -- 172.21.15.90:0/3021411347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 msgr2=0x5624ce0f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 -- 172.21.15.90:0/3021411347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624cdfe78b0 con 0x5624ce0eecf0 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 0x5624ce0ef110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006010 tx=0x7ffac80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8d27f5dc41c43b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffac2000640 1 -- 172.21.15.90:0/3021411347 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8031450 con 0x5624ce0eecf0 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffac2000640 1 -- 172.21.15.90:0/3021411347 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac8036040 con 0x5624ce0eecf0 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.482+0000 7ffac2000640 1 -- 172.21.15.90:0/3021411347 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8004580 con 0x5624ce0eecf0 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffacd600640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 0x7ffab80b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffacd600640 1 -- 172.21.15.90:0/1468842167 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffac40e2be0 con 0x7ffad0104b10 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffad5000640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 0x7ffab80b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 -- 172.21.15.90:0/3021411347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 msgr2=0x5624ce0ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 0x5624ce0ef110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006010 tx=0x7ffac80093e0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 -- 172.21.15.90:0/3021411347 shutdown_connections 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0f4850 0x5624ce0f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 --2- 172.21.15.90:0/3021411347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0eecf0 0x5624ce0ef110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 -- 172.21.15.90:0/3021411347 >> 172.21.15.90:0/3021411347 conn(0x5624ce0ea360 msgr2=0x5624ce0ea760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 -- 172.21.15.90:0/3021411347 shutdown_connections 2024-06-02T15:43:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.483+0000 7ffada855740 1 -- 172.21.15.90:0/3021411347 wait complete. 2024-06-02T15:43:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.484+0000 7ffad5000640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 0x7ffab80b4ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffac40a9d80 tx=0x7ffac40e6620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.484+0000 7ffada855740 1 -- start start 2024-06-02T15:43:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.484+0000 7ffada855740 1 Processor -- start 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.484+0000 7ffada855740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.484+0000 7ffada855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffada855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffada855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5624cdd34790 con 0x5624ce0f4850 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffada855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5624cdff6bc0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffada855740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5624ce0f77a0 con 0x5624ce0eecf0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffada855740 10 client.0 _ll_register_callbacks cb 0x5624cdfd4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffad5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffad5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60368/0 (socket says 172.21.15.90:60368) 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffad5a00640 1 -- 172.21.15.90:0/2372056151 learned_addr learned my addr 172.21.15.90:0/2372056151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffad5a00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 -- 172.21.15.90:0/2372056151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 msgr2=0x5624ce0f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 -- 172.21.15.90:0/2372056151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 msgr2=0x5624ce0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.485+0000 7ffacfe00640 1 -- 172.21.15.90:0/2372056151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624cdd63210 con 0x5624ce0ef6e0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f8f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffad019c1c0 tx=0x7ffabc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cbb77ed7c8ff4bf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:43:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00f8c0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffac8036040 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffabc00fa60 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffabc00fc60 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffada855740 10 client.25259 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.486+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624cde2f620 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac8036040 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ffabc01c420 con 0x5624ce0ef6e0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 10 client.25259 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 20 client.25259 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce1045b0 0x5624ce1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 4 client.25259 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624ce060ae0 con 0x5624ce1045b0 2024-06-02T15:43:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.487+0000 7ffad5a00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce1045b0 0x5624ce1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.488+0000 7ffab6a00640 10 client.25259 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.488+0000 7ffad5a00640 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce1045b0 0x5624ce1069f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512c0492fd11e55a server_cookie=5261996541b6a2f1 in_seq=0 out_seq=0 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffab6a00640 1 -- 172.21.15.90:0/2372056151 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5624ce060ae0 con 0x5624ce1045b0 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffab6a00640 10 client.25259 handle_command_reply: tid=0 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 client.25259 shutdown 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 msgr2=0x5624ce0f8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f8f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffad019c1c0 tx=0x7ffabc012b60 comp rx=0 tx=0).stop 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 shutdown_connections 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce1045b0 0x5624ce1069f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0f8f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 --2- 172.21.15.90:0/2372056151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffab6a00640 0 client.25259 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 >> 172.21.15.90:0/2372056151 conn(0x5624ce0ea360 msgr2=0x5624ce0e94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 shutdown_connections 2024-06-02T15:43:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 1 -- 172.21.15.90:0/2372056151 wait complete. 2024-06-02T15:43:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.489+0000 7ffada855740 20 client.25259 trim_cache size 0 max 16384 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 Processor -- start 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 -- start start 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01affd0 con 0x5624ce0f4850 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01ad8a0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffada855740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01ab6f0 con 0x5624ce0eecf0 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffad5a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37412/0 (socket says 172.21.15.90:37412) 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.506+0000 7ffacfe00640 1 -- 172.21.15.90:0/2044035361 learned_addr learned my addr 172.21.15.90:0/2044035361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffad5a00640 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 -- 172.21.15.90:0/2044035361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 msgr2=0x5624ce0f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 -- 172.21.15.90:0/2044035361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 msgr2=0x5624ce0f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 -- 172.21.15.90:0/2044035361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624ce0ea190 con 0x5624ce0ef6e0 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffad5a00640 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffacfe00640 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009e00 tx=0x7ffabc00ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a0e27b286994a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffac1600640 1 -- 172.21.15.90:0/2044035361 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc01cb50 con 0x5624ce0ef6e0 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffac1600640 1 -- 172.21.15.90:0/2044035361 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc01ccf0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.507+0000 7ffac1600640 1 -- 172.21.15.90:0/2044035361 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0035f0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- 172.21.15.90:0/2044035361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 msgr2=0x5624ce0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009e00 tx=0x7ffabc00ff90 comp rx=0 tx=0).stop 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- 172.21.15.90:0/2044035361 shutdown_connections 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce0f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fff00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 --2- 172.21.15.90:0/2044035361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0eecf0 0x5624ce0f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- 172.21.15.90:0/2044035361 >> 172.21.15.90:0/2044035361 conn(0x5624ce0ea360 msgr2=0x5624ce0f3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- 172.21.15.90:0/2044035361 shutdown_connections 2024-06-02T15:43:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- 172.21.15.90:0/2044035361 wait complete. 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.508+0000 7ffada855740 1 -- start start 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 Processor -- start 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0eecf0 0x5624ce0fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce175990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01abd60 con 0x5624ce0f4850 2024-06-02T15:43:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01ad1d0 con 0x5624ce0eecf0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01ac430 con 0x5624ce0ef6e0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffada855740 10 client.0 _ll_register_callbacks cb 0x7ffabc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffad5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce175990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:50326/0 (socket says 172.21.15.90:50326) 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffacfe00640 1 -- 172.21.15.90:0/3419473452 learned_addr learned my addr 172.21.15.90:0/3419473452 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.509+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0eecf0 0x5624ce0fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 -- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0eecf0 msgr2=0x5624ce0fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0eecf0 0x5624ce0fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 -- 172.21.15.90:0/3419473452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 msgr2=0x5624ce175990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce175990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 -- 172.21.15.90:0/3419473452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624ce175f80 con 0x5624ce0ef6e0 2024-06-02T15:43:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffacfe00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffabc01c6d0 tx=0x7ffabc01c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6c543d9a6f3681 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffab6a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc02f070 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffabc01ccf0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.510+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ffabc00f9a0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffabc03a050 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffada855740 10 client.25237 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624ce177200 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffabc01ccf0 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ffabc029030 con 0x5624ce0ef6e0 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffada855740 10 client.25237 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffada855740 10 client.25237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.511+0000 7ffada855740 20 client.25237 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce177b20 0x5624ce179f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffada855740 4 client.25237 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:43:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624ce05a2c0 con 0x5624ce177b20 2024-06-02T15:43:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce177b20 0x5624ce179f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffad5a00640 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce177b20 0x5624ce179f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9671eb3e4851616e server_cookie=df087c8ad0329672 in_seq=0 out_seq=0 2024-06-02T15:43:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.512+0000 7ffab6a00640 10 client.25237 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.524+0000 7ffab6a00640 1 -- 172.21.15.90:0/3419473452 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5624ce05a2c0 con 0x5624ce177b20 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.524+0000 7ffab6a00640 10 client.25237 handle_command_reply: tid=0 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.524+0000 7ffada855740 1 client.25237 shutdown 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 msgr2=0x5624ce0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffabc01c6d0 tx=0x7ffabc01c700 comp rx=0 tx=0).stop 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 shutdown_connections 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5624ce177b20 0x5624ce179f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5624ce0f4850 0x5624ce175990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5624ce0ef6e0 0x5624ce0fd460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffab6a00640 0 client.25237 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 --2- 172.21.15.90:0/3419473452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5624ce0eecf0 0x5624ce0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 >> 172.21.15.90:0/3419473452 conn(0x5624ce0ea360 msgr2=0x5624ce0ed040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 shutdown_connections 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 1 -- 172.21.15.90:0/3419473452 wait complete. 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.525+0000 7ffada855740 20 client.25237 trim_cache size 0 max 16384 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:43:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 msgr2=0x7ffab80b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 0x7ffab80b4ca0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffac40a9d80 tx=0x7ffac40e6620 comp rx=0 tx=0).stop 2024-06-02T15:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 msgr2=0x7ffad019acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9c0 tx=0x7ffac400ee80 comp rx=0 tx=0).stop 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 shutdown_connections 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ffab80b27e0 0x7ffab80b4ca0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad01066c0 0x7ffad01a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffad0105d00 0x7ffad019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 --2- 172.21.15.90:0/1468842167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffad0104b10 0x7ffad019acc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 >> 172.21.15.90:0/1468842167 conn(0x7ffad01000c0 msgr2=0x7ffad01090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 shutdown_connections 2024-06-02T15:43:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:45.526+0000 7ffad6400640 1 -- 172.21.15.90:0/1468842167 wait complete. 2024-06-02T15:43:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:45 smithi090 ceph-mon[31595]: pgmap v493: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:45 smithi120 ceph-mon[36451]: pgmap v493: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:45.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:45 smithi135 ceph-mon[36839]: pgmap v493: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-06-02T15:43:47.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:47 smithi090 ceph-mon[31595]: pgmap v494: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-02T15:43:47.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:47 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:43:47.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:47 smithi120 ceph-mon[36451]: pgmap v494: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-02T15:43:47.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:47 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:43:47.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:47 smithi135 ceph-mon[36839]: pgmap v494: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-06-02T15:43:47.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:47 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:43:49.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:49 smithi090 ceph-mon[31595]: pgmap v495: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:43:49.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:49 smithi120 ceph-mon[36451]: pgmap v495: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:43:49.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:49 smithi135 ceph-mon[36839]: pgmap v495: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-06-02T15:43:51.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:51 smithi090 ceph-mon[31595]: pgmap v496: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:51.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:51 smithi120 ceph-mon[36451]: pgmap v496: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:51.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:51 smithi135 ceph-mon[36839]: pgmap v496: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:53 smithi090 ceph-mon[31595]: pgmap v497: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-02T15:43:53.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:53 smithi120 ceph-mon[36451]: pgmap v497: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-02T15:43:53.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:53 smithi135 ceph-mon[36839]: pgmap v497: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-06-02T15:43:55.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:55 smithi120 ceph-mon[36451]: pgmap v498: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:55 smithi090 ceph-mon[31595]: pgmap v498: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:55.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:55 smithi135 ceph-mon[36839]: pgmap v498: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:57.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:57 smithi090 ceph-mon[31595]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:57.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:57 smithi120 ceph-mon[36451]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:57.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:57 smithi135 ceph-mon[36839]: pgmap v499: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-06-02T15:43:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 -- 172.21.15.90:0/1581395554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53401090e0 msgr2=0x7f5340109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 --2- 172.21.15.90:0/1581395554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53401090e0 0x7f5340109500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5330009860 tx=0x7f533002a340 comp rx=0 tx=0).stop 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 -- 172.21.15.90:0/1581395554 shutdown_connections 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 --2- 172.21.15.90:0/1581395554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f534010ac90 0x7f5340111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 --2- 172.21.15.90:0/1581395554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 0x7f534010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 --2- 172.21.15.90:0/1581395554 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53401090e0 0x7f5340109500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 -- 172.21.15.90:0/1581395554 >> 172.21.15.90:0/1581395554 conn(0x7f53400780a0 msgr2=0x7f534007a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 -- 172.21.15.90:0/1581395554 shutdown_connections 2024-06-02T15:43:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.668+0000 7f5345200640 1 -- 172.21.15.90:0/1581395554 wait complete. 2024-06-02T15:43:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.669+0000 7f5345200640 1 Processor -- start 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 -- start start 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 0x7f534019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534010ac90 0x7f53401a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114be0 con 0x7f53401090e0 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114a60 con 0x7f534010ac90 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f5345200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5340114d60 con 0x7f534010a2d0 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f533de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f533de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39768/0 (socket says 172.21.15.90:39768) 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f533de00640 1 -- 172.21.15.90:0/1211891885 learned_addr learned my addr 172.21.15.90:0/1211891885 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f533d400640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 0x7f534019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.670+0000 7f533e800640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534010ac90 0x7f53401a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 -- 172.21.15.90:0/1211891885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 msgr2=0x7f534019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 0x7f534019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 -- 172.21.15.90:0/1211891885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534010ac90 msgr2=0x7f53401a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534010ac90 0x7f53401a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 -- 172.21.15.90:0/1211891885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53401a6e70 con 0x7f53401090e0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f533de00640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53300057c0 tx=0x7f5330032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f5336a00640 1 -- 172.21.15.90:0/1211891885 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330037040 con 0x7f53401090e0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f5336a00640 1 -- 172.21.15.90:0/1211891885 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5330030cd0 con 0x7f53401090e0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53401a7100 con 0x7f53401090e0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.671+0000 7f5336a00640 1 -- 172.21.15.90:0/1211891885 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330005310 con 0x7f53401090e0 2024-06-02T15:43:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.672+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53401a75e0 con 0x7f53401090e0 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 Processor -- start 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 -- start start 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdca1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612bdc0bb90 con 0x5612bdc9bb30 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612bdbfb730 con 0x5612bdc9c520 2024-06-02T15:43:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f534816d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5612bdbff770 con 0x5612bdca1690 2024-06-02T15:43:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f5336a00640 1 -- 172.21.15.90:0/1211891885 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5330043020 con 0x7f53401090e0 2024-06-02T15:43:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.675+0000 7f533e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:59484/0 (socket says 172.21.15.90:59484) 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 -- 172.21.15.90:0/2528760457 learned_addr learned my addr 172.21.15.90:0/2528760457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39772/0 (socket says 172.21.15.90:39772) 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533de00640 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdca1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 -- 172.21.15.90:0/2528760457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 msgr2=0x5612bdca1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdca1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 -- 172.21.15.90:0/2528760457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 msgr2=0x5612bdc9bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 -- 172.21.15.90:0/2528760457 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bdb948b0 con 0x5612bdca1690 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533e800640 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.676+0000 7f533d400640 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f532c00bd10 tx=0x7f532c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93f376ce5a597b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f532b400640 1 -- 172.21.15.90:0/2528760457 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c005220 con 0x5612bdca1690 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f532b400640 1 -- 172.21.15.90:0/2528760457 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532c004c50 con 0x5612bdca1690 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f532b400640 1 -- 172.21.15.90:0/2528760457 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c011820 con 0x5612bdca1690 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f5336a00640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 0x7f53240b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f5336a00640 1 -- 172.21.15.90:0/1211891885 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5330101d10 con 0x7f53401090e0 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f534816d740 1 -- 172.21.15.90:0/2528760457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 msgr2=0x5612bdca3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f534816d740 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f532c00bd10 tx=0x7f532c0107a0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.677+0000 7f533e800640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 0x7f53240b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 -- 172.21.15.90:0/2528760457 shutdown_connections 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdca3b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdca1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 --2- 172.21.15.90:0/2528760457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdc9bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 -- 172.21.15.90:0/2528760457 >> 172.21.15.90:0/2528760457 conn(0x5612bdc971a0 msgr2=0x5612bdc975a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 -- 172.21.15.90:0/2528760457 shutdown_connections 2024-06-02T15:43:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 -- 172.21.15.90:0/2528760457 wait complete. 2024-06-02T15:43:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 -- start start 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f533e800640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 0x7f53240b4de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5338009e20 tx=0x7f5338010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.678+0000 7f534816d740 1 Processor -- start 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdca58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 0x5612bdca5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 0x5612bdcace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612bd8e1790 con 0x5612bdc9bb30 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5612bdba3bc0 con 0x5612bdca1690 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5612bdca4640 con 0x5612bdc9c520 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f534816d740 10 client.0 _ll_register_callbacks cb 0x5612bdb81400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f533e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 0x5612bdcace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f533de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdca58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f533e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 0x5612bdcace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:49532/0 (socket says 172.21.15.90:49532) 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f533e800640 1 -- 172.21.15.90:0/3060920472 learned_addr learned my addr 172.21.15.90:0/3060920472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.679+0000 7f533d400640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 0x5612bdca5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 -- 172.21.15.90:0/3060920472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 msgr2=0x5612bdca5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 0x5612bdca5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 -- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 msgr2=0x5612bdcace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 0x5612bdcace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 -- 172.21.15.90:0/3060920472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bd910210 con 0x5612bdc9bb30 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533d400640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 0x5612bdca5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f533de00640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdca58f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5612bdc9cf90 tx=0x7f53300c8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcfffca0206ad73 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f531fe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:43:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53300c9620 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f53300c5510 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c004c50 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53300c8f20 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f534816d740 10 client.15156 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.680+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612bd9dc620 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532c004c50 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5330107030 con 0x5612bdc9bb30 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 10 client.15156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 20 client.15156 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdcb1450 0x5612bdcb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 4 client.15156 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.681+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612bdc0dae0 con 0x5612bdcb1450 2024-06-02T15:43:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.682+0000 7f533d400640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdcb1450 0x5612bdcb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.682+0000 7f533d400640 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdcb1450 0x5612bdcb38a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6b95baaebb7d14 server_cookie=b558bec7554a0360 in_seq=0 out_seq=0 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.682+0000 7f531fe00640 10 client.15156 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f531fe00640 1 -- 172.21.15.90:0/3060920472 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612bdc0dae0 con 0x5612bdcb1450 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f531fe00640 10 client.15156 handle_command_reply: tid=0 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 client.15156 shutdown 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 msgr2=0x5612bdca58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdca58f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5612bdc9cf90 tx=0x7f53300c8930 comp rx=0 tx=0).stop 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 shutdown_connections 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdcb1450 0x5612bdcb38a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdca1690 0x5612bdcace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9c520 0x5612bdca5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 --2- 172.21.15.90:0/3060920472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdca58f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f531fe00640 0 client.15156 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.683+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 >> 172.21.15.90:0/3060920472 conn(0x5612bdc971a0 msgr2=0x5612bdc96310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.684+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 shutdown_connections 2024-06-02T15:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.684+0000 7f534816d740 1 -- 172.21.15.90:0/3060920472 wait complete. 2024-06-02T15:43:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.684+0000 7f534816d740 20 client.15156 trim_cache size 0 max 16384 2024-06-02T15:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.700+0000 7f534816d740 1 Processor -- start 2024-06-02T15:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.700+0000 7f534816d740 1 -- start start 2024-06-02T15:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.700+0000 7f534816d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f534816d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f534816d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdcb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f534816d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f53401b4540 con 0x5612bdc9bb30 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f534816d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53401b1e10 con 0x5612bdc9c520 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f534816d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53401afc60 con 0x5612bdca1690 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39794/0 (socket says 172.21.15.90:39794) 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 -- 172.21.15.90:0/4038254252 learned_addr learned my addr 172.21.15.90:0/4038254252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533e800640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdcb9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533d400640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 -- 172.21.15.90:0/4038254252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 msgr2=0x5612bdcb9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdcb9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 -- 172.21.15.90:0/4038254252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 msgr2=0x5612bdcace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 -- 172.21.15.90:0/4038254252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bdc96fd0 con 0x5612bdc9bb30 2024-06-02T15:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533d400640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:43:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.701+0000 7f533de00640 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53300095f0 tx=0x7f5330105280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a30b0d51ca75c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f532aa00640 1 -- 172.21.15.90:0/4038254252 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330030e80 con 0x5612bdc9bb30 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f532aa00640 1 -- 172.21.15.90:0/4038254252 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53300c9620 con 0x5612bdc9bb30 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f532aa00640 1 -- 172.21.15.90:0/4038254252 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53300c4b60 con 0x5612bdc9bb30 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 -- 172.21.15.90:0/4038254252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 msgr2=0x5612bdcb9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53300095f0 tx=0x7f5330105280 comp rx=0 tx=0).stop 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 -- 172.21.15.90:0/4038254252 shutdown_connections 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdca1690 0x5612bdcb9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 --2- 172.21.15.90:0/4038254252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdc9bb30 0x5612bdcb9150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.702+0000 7f534816d740 1 -- 172.21.15.90:0/4038254252 >> 172.21.15.90:0/4038254252 conn(0x5612bdc971a0 msgr2=0x5612bdc91dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 -- 172.21.15.90:0/4038254252 shutdown_connections 2024-06-02T15:43:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 -- 172.21.15.90:0/4038254252 wait complete. 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 -- start start 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 Processor -- start 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9bb30 0x5612bdca9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.703+0000 7f534816d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f53401b02d0 con 0x5612bdca1690 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f534816d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f53401b1740 con 0x5612bdc9c520 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f534816d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53401b09a0 con 0x5612bdc9bb30 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f534816d740 10 client.0 _ll_register_callbacks cb 0x7f533002af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533de00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9bb30 0x5612bdca9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39802/0 (socket says 172.21.15.90:39802) 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 -- 172.21.15.90:0/4077396390 learned_addr learned my addr 172.21.15.90:0/4077396390 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:43:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533d400640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 -- 172.21.15.90:0/4077396390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9bb30 msgr2=0x5612bdca9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9bb30 0x5612bdca9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 -- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 msgr2=0x5612bdcaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.704+0000 7f533e800640 1 -- 172.21.15.90:0/4077396390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bdcaa740 con 0x5612bdca1690 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f533e800640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53380045f0 tx=0x7f5338004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78fe549b1efeedf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f531fe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533802a030 con 0x5612bdca1690 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f533800dc60 con 0x5612bdca1690 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53300c9620 con 0x5612bdca1690 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5338036050 con 0x5612bdca1690 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f534816d740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-06-02T15:43:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612bdd1b700 con 0x5612bdca1690 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.705+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53300c9620 con 0x5612bdca1690 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f533800b040 con 0x5612bdca1690 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 10 client.15162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 20 client.15162 populate_metadata read hostname 'smithi090' 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdd1c110 0x5612bdd1e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 4 client.15162 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:43:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612bdc072c0 con 0x5612bdd1c110 2024-06-02T15:43:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.706+0000 7f533de00640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdd1c110 0x5612bdd1e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:43:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.707+0000 7f533de00640 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdd1c110 0x5612bdd1e530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f6e74cc9bd83c server_cookie=94e9bd58277727ba in_seq=0 out_seq=0 2024-06-02T15:43:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.707+0000 7f531fe00640 10 client.15162 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.710+0000 7f531fe00640 1 -- 172.21.15.90:0/4077396390 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5612bdc072c0 con 0x5612bdd1c110 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.710+0000 7f531fe00640 10 client.15162 handle_command_reply: tid=0 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.710+0000 7f534816d740 1 client.15162 shutdown 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 msgr2=0x5612bdd1a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53380045f0 tx=0x7f5338004ab0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 shutdown_connections 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5612bdd1c110 0x5612bdd1e530 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612bdca1690 0x5612bdd1a010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5612bdc9c520 0x5612bdcaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 --2- 172.21.15.90:0/4077396390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612bdc9bb30 0x5612bdca9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f531fe00640 0 client.15162 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 >> 172.21.15.90:0/4077396390 conn(0x5612bdc971a0 msgr2=0x5612bdca5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 shutdown_connections 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 1 -- 172.21.15.90:0/4077396390 wait complete. 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.711+0000 7f534816d740 20 client.15162 trim_cache size 0 max 16384 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:43:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:43:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:43:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 msgr2=0x7f53240b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 0x7f53240b4de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5338009e20 tx=0x7f5338010040 comp rx=0 tx=0).stop 2024-06-02T15:43:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 msgr2=0x7f534019f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:43:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53300057c0 tx=0x7f5330032eb0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 shutdown_connections 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53240b2920 0x7f53240b4de0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f534010ac90 0x7f53401a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f534010a2d0 0x7f534019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 --2- 172.21.15.90:0/1211891885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53401090e0 0x7f534019f250 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 >> 172.21.15.90:0/1211891885 conn(0x7f53400780a0 msgr2=0x7f534010d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 shutdown_connections 2024-06-02T15:43:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:43:58.712+0000 7f5345200640 1 -- 172.21.15.90:0/1211891885 wait complete. 2024-06-02T15:43:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:43:59 smithi090 ceph-mon[31595]: pgmap v500: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T15:43:59.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:43:59 smithi120 ceph-mon[36451]: pgmap v500: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T15:43:59.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:43:59 smithi135 ceph-mon[36839]: pgmap v500: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T15:44:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:01 smithi090 ceph-mon[31595]: pgmap v501: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:01.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:01 smithi120 ceph-mon[36451]: pgmap v501: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:01.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:01 smithi135 ceph-mon[36839]: pgmap v501: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:03 smithi120 ceph-mon[36451]: pgmap v502: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:44:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:03 smithi135 ceph-mon[36839]: pgmap v502: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:44:03.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:03 smithi090 ceph-mon[31595]: pgmap v502: 129 pgs: 129 active+clean; 657 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-06-02T15:44:05.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:05 smithi120 ceph-mon[36451]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:05.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:05 smithi135 ceph-mon[36839]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:05.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:05 smithi090 ceph-mon[31595]: pgmap v503: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:07.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:07 smithi120 ceph-mon[36451]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:07.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:07 smithi135 ceph-mon[36839]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:07.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:07 smithi090 ceph-mon[31595]: pgmap v504: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:09 smithi120 ceph-mon[36451]: pgmap v505: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:09 smithi135 ceph-mon[36839]: pgmap v505: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:09.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:09 smithi090 ceph-mon[31595]: pgmap v505: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:11.326 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:44:11.781 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:11.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:11 smithi120 ceph-mon[36451]: pgmap v506: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:11.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:11 smithi135 ceph-mon[36839]: pgmap v506: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 -- 172.21.15.90:0/2808687621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50075450 msgr2=0x7f8e500758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2808687621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50075450 0x7f8e500758b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009900 tx=0x7f8e3c02a480 comp rx=0 tx=0).stop 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 -- 172.21.15.90:0/2808687621 shutdown_connections 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2808687621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e5010a800 0x7f8e5010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2808687621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50075450 0x7f8e500758b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2808687621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e50076a30 0x7f8e50074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 -- 172.21.15.90:0/2808687621 >> 172.21.15.90:0/2808687621 conn(0x7f8e500fdee0 msgr2=0x7f8e50100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 -- 172.21.15.90:0/2808687621 shutdown_connections 2024-06-02T15:44:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.855+0000 7f8e4f200640 1 -- 172.21.15.90:0/2808687621 wait complete. 2024-06-02T15:44:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.856+0000 7f8e4f200640 1 Processor -- start 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 -- start start 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 0x7f8e5019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 0x7f8e5019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e5010a800 0x7f8e501a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50110680 con 0x7f8e50076a30 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50110500 con 0x7f8e5010a800 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4f200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50110800 con 0x7f8e50075450 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 0x7f8e5019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4e800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e5010a800 0x7f8e501a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 0x7f8e5019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49544/0 (socket says 172.21.15.90:49544) 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4d400640 1 -- 172.21.15.90:0/2138090681 learned_addr learned my addr 172.21.15.90:0/2138090681 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.857+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 0x7f8e5019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 -- 172.21.15.90:0/2138090681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e5010a800 msgr2=0x7f8e501a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e5010a800 0x7f8e501a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 -- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 msgr2=0x7f8e5019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 0x7f8e5019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 -- 172.21.15.90:0/2138090681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e501a2900 con 0x7f8e50075450 2024-06-02T15:44:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 0x7f8e5019acf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e46a00640 1 -- 172.21.15.90:0/2138090681 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000ccc0 con 0x7f8e50075450 2024-06-02T15:44:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e46a00640 1 -- 172.21.15.90:0/2138090681 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e400108e0 con 0x7f8e50075450 2024-06-02T15:44:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e501a2bf0 con 0x7f8e50075450 2024-06-02T15:44:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.858+0000 7f8e46a00640 1 -- 172.21.15.90:0/2138090681 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40018880 con 0x7f8e50075450 2024-06-02T15:44:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.859+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a3140 con 0x7f8e50075450 2024-06-02T15:44:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 Processor -- start 2024-06-02T15:44:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 -- start start 2024-06-02T15:44:11.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55890f0acb90 con 0x55890f13ccf0 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55890f09c730 con 0x55890f142850 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890f0a0770 con 0x55890f13d6e0 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e46a00640 1 -- 172.21.15.90:0/2138090681 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8e40010430 con 0x7f8e50075450 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.862+0000 7f8e4e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49548/0 (socket says 172.21.15.90:49548) 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 -- 172.21.15.90:0/777789760 learned_addr learned my addr 172.21.15.90:0/777789760 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4de00640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 -- 172.21.15.90:0/777789760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 msgr2=0x55890f1421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f1421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 -- 172.21.15.90:0/777789760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 msgr2=0x55890f144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4e800640 1 -- 172.21.15.90:0/777789760 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890f0358b0 con 0x55890f13ccf0 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4de00640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f1421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.863+0000 7f8e4d400640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.864+0000 7f8e4e800640 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009fd0 tx=0x7f8e480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da931b88f5a0814b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.864+0000 7f8e3b400640 1 -- 172.21.15.90:0/777789760 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48004d20 con 0x55890f13ccf0 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.864+0000 7f8e3b400640 1 -- 172.21.15.90:0/777789760 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e48004050 con 0x55890f13ccf0 2024-06-02T15:44:11.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.864+0000 7f8e3b400640 1 -- 172.21.15.90:0/777789760 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48005280 con 0x55890f13ccf0 2024-06-02T15:44:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.864+0000 7f8e46a00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 0x7f8e340b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e46a00640 1 -- 172.21.15.90:0/2138090681 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8e400e29d0 con 0x7f8e50075450 2024-06-02T15:44:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 -- 172.21.15.90:0/777789760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 msgr2=0x55890f13d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009fd0 tx=0x7f8e480118d0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 0x7f8e340b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 -- 172.21.15.90:0/777789760 shutdown_connections 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f1421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/777789760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f13d110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 -- 172.21.15.90:0/777789760 >> 172.21.15.90:0/777789760 conn(0x55890f138360 msgr2=0x55890f138760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 -- 172.21.15.90:0/777789760 shutdown_connections 2024-06-02T15:44:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.865+0000 7f8e57f6f740 1 -- 172.21.15.90:0/777789760 wait complete. 2024-06-02T15:44:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e4de00640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 0x7f8e340b4ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e400a6e70 tx=0x7f8e400e8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 -- start start 2024-06-02T15:44:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 Processor -- start 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13ccf0 0x55890f146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f14e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55890ed82790 con 0x55890f13d6e0 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55890f044bc0 con 0x55890f142850 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890f145800 con 0x55890f13ccf0 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e57f6f740 10 client.0 _ll_register_callbacks cb 0x55890f022400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.866+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49564/0 (socket says 172.21.15.90:49564) 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 -- 172.21.15.90:0/1393167882 learned_addr learned my addr 172.21.15.90:0/1393167882 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13ccf0 0x55890f146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 -- 172.21.15.90:0/1393167882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13ccf0 msgr2=0x55890f146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13ccf0 0x55890f146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 -- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 msgr2=0x55890f14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 -- 172.21.15.90:0/1393167882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890edb1210 con 0x55890f13d6e0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02aa70 tx=0x7f8e3c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e8b4c377c6ab32 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.867+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c035040 con 0x55890f13d6e0 2024-06-02T15:44:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8e3c003a00 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e48004050 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c04e050 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e57f6f740 10 client.15174 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55890ee7d620 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e48004050 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8e3c03c440 con 0x55890f13d6e0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e57f6f740 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.868+0000 7f8e57f6f740 10 client.15174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e57f6f740 20 client.15174 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1527f0 0x55890f154be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e57f6f740 4 client.15174 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890f0aeae0 con 0x55890f1527f0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1527f0 0x55890f154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1527f0 0x55890f154be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a594556fc49b77c3 server_cookie=10e81487a59798a9 in_seq=0 out_seq=0 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.869+0000 7f8e2fe00640 10 client.15174 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.870+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1393167882 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55890f0aeae0 con 0x55890f1527f0 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.870+0000 7f8e2fe00640 10 client.15174 handle_command_reply: tid=0 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.870+0000 7f8e57f6f740 1 client.15174 shutdown 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 msgr2=0x55890f1470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c02aa70 tx=0x7f8e3c0043b0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 shutdown_connections 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1527f0 0x55890f154be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f1470e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1393167882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13ccf0 0x55890f146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 >> 172.21.15.90:0/1393167882 conn(0x55890f138360 msgr2=0x55890f1374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e2fe00640 0 client.15174 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 shutdown_connections 2024-06-02T15:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1393167882 wait complete. 2024-06-02T15:44:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.871+0000 7f8e57f6f740 20 client.15174 trim_cache size 0 max 16384 2024-06-02T15:44:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:11 smithi090 ceph-mon[31595]: pgmap v506: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 Processor -- start 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 -- start start 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 0x55890f141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f142850 0x55890f136c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501afeb0 con 0x55890f13d6e0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ad780 con 0x55890f13ccf0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ab5d0 con 0x55890f142850 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49572/0 (socket says 172.21.15.90:49572) 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.888+0000 7f8e4d400640 1 -- 172.21.15.90:0/1705808145 learned_addr learned my addr 172.21.15.90:0/1705808145 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 0x55890f141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f142850 0x55890f136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 -- 172.21.15.90:0/1705808145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f142850 msgr2=0x55890f136c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f142850 0x55890f136c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 -- 172.21.15.90:0/1705808145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 msgr2=0x55890f141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 0x55890f141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 -- 172.21.15.90:0/1705808145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890f147b90 con 0x55890f13d6e0 2024-06-02T15:44:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 0x55890f141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c006010 tx=0x7f8e3c03ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0235b94a218cc00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e3aa00640 1 -- 172.21.15.90:0/1705808145 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c035030 con 0x55890f13d6e0 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e3aa00640 1 -- 172.21.15.90:0/1705808145 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3c03c640 con 0x55890f13d6e0 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.889+0000 7f8e3aa00640 1 -- 172.21.15.90:0/1705808145 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c003a60 con 0x55890f13d6e0 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1705808145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 msgr2=0x55890f14e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c006010 tx=0x7f8e3c03ef90 comp rx=0 tx=0).stop 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1705808145 shutdown_connections 2024-06-02T15:44:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f142850 0x55890f136c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13d6e0 0x55890f14e0e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1705808145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f13ccf0 0x55890f141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1705808145 >> 172.21.15.90:0/1705808145 conn(0x55890f138360 msgr2=0x55890f13ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.890+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1705808145 shutdown_connections 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1705808145 wait complete. 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 -- start start 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 Processor -- start 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f1bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501abc40 con 0x55890f13ccf0 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ad0b0 con 0x55890f142850 2024-06-02T15:44:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ac310 con 0x55890f13d6e0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.891+0000 7f8e57f6f740 10 client.0 _ll_register_callbacks cb 0x7f8e3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55934/0 (socket says 172.21.15.90:55934) 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 -- 172.21.15.90:0/1442157945 learned_addr learned my addr 172.21.15.90:0/1442157945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49574/0 (socket says 172.21.15.90:49574) 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f1bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 -- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 msgr2=0x55890f1bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f1bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 -- 172.21.15.90:0/1442157945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 msgr2=0x55890f14b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 -- 172.21.15.90:0/1442157945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890f1bb8b0 con 0x55890f13d6e0 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e4d400640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005a60 tx=0x7f8e3c003a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=514b4d04082ea924 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.892+0000 7f8e2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c035070 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e3c03c640 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8e3c003db0 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c05f050 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e57f6f740 10 client.25273 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55890f1bcbc0 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.893+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e3c03c640 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8e3c046430 con 0x55890f13d6e0 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 10 client.25273 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 10 client.25273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 20 client.25273 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1bd560 0x55890f1bf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 4 client.25273 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55890f0a82c0 con 0x55890f1bd560 2024-06-02T15:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.894+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1bd560 0x55890f1bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.895+0000 7f8e2fe00640 10 client.25273 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.895+0000 7f8e4e800640 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1bd560 0x55890f1bf980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df560c09970c0e78 server_cookie=fc83194813d93d9c in_seq=0 out_seq=0 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e2fe00640 1 -- 172.21.15.90:0/1442157945 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55890f0a82c0 con 0x55890f1bd560 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e2fe00640 10 client.25273 handle_command_reply: tid=0 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 client.25273 shutdown 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 msgr2=0x55890f14b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005a60 tx=0x7f8e3c003a30 comp rx=0 tx=0).stop 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 shutdown_connections 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55890f1bd560 0x55890f1bf980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55890f142850 0x55890f1bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f13d6e0 0x55890f14b660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 --2- 172.21.15.90:0/1442157945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55890f13ccf0 0x55890f14b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e2fe00640 0 client.25273 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.902+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 >> 172.21.15.90:0/1442157945 conn(0x55890f138360 msgr2=0x55890f13ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 shutdown_connections 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e57f6f740 1 -- 172.21.15.90:0/1442157945 wait complete. 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e57f6f740 20 client.25273 trim_cache size 0 max 16384 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:44:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 msgr2=0x7f8e340b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 0x7f8e340b4ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e400a6e70 tx=0x7f8e400e8530 comp rx=0 tx=0).stop 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 msgr2=0x7f8e5019acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.903+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 0x7f8e5019acf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000eeb0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 shutdown_connections 2024-06-02T15:44:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8e340b27e0 0x7f8e340b4ca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8e5010a800 0x7f8e501a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e50076a30 0x7f8e5019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 --2- 172.21.15.90:0/2138090681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e50075450 0x7f8e5019acf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 >> 172.21.15.90:0/2138090681 conn(0x7f8e500fdee0 msgr2=0x7f8e50106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 shutdown_connections 2024-06-02T15:44:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:11.904+0000 7f8e4f200640 1 -- 172.21.15.90:0/2138090681 wait complete. 2024-06-02T15:44:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:13 smithi120 ceph-mon[36451]: pgmap v507: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-02T15:44:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:13 smithi135 ceph-mon[36839]: pgmap v507: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-02T15:44:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:13 smithi090 ceph-mon[31595]: pgmap v507: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-06-02T15:44:15.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:15 smithi120 ceph-mon[36451]: pgmap v508: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-02T15:44:15.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:15 smithi135 ceph-mon[36839]: pgmap v508: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-02T15:44:15.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:15 smithi090 ceph-mon[31595]: pgmap v508: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-06-02T15:44:16.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 -- 172.21.15.90:0/1951729798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf4104ff0 msgr2=0x7faaf410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 --2- 172.21.15.90:0/1951729798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf4104ff0 0x7faaf410d4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faae0009990 tx=0x7faae0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 -- 172.21.15.90:0/1951729798 shutdown_connections 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 --2- 172.21.15.90:0/1951729798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf410ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 --2- 172.21.15.90:0/1951729798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf41046d0 0x7faaf4104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 --2- 172.21.15.90:0/1951729798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf4104ff0 0x7faaf410d4e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.678+0000 7faafae00700 1 -- 172.21.15.90:0/1951729798 >> 172.21.15.90:0/1951729798 conn(0x7faaf40fde60 msgr2=0x7faaf4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.679+0000 7faafae00700 1 -- 172.21.15.90:0/1951729798 shutdown_connections 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.679+0000 7faafae00700 1 -- 172.21.15.90:0/1951729798 wait complete. 2024-06-02T15:44:16.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 Processor -- start 2024-06-02T15:44:16.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 -- start start 2024-06-02T15:44:16.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 0x7faaf419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 0x7faaf419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41136b0 con 0x7faaf41046d0 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4113530 con 0x7faaf4104ff0 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.680+0000 7faafae00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4113830 con 0x7faaf410dab0 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faafa400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 0x7faaf419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faafa400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55952/0 (socket says 172.21.15.90:55952) 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faafa400700 1 -- 172.21.15.90:0/1167755602 learned_addr learned my addr 172.21.15.90:0/1167755602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:16.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 0x7faaf419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9000700 1 -- 172.21.15.90:0/1167755602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 msgr2=0x7faaf41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9000700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9000700 1 -- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 msgr2=0x7faaf419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.681+0000 7faaf9000700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 0x7faaf419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf9000700 1 -- 172.21.15.90:0/1167755602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419a110 con 0x7faaf4104ff0 2024-06-02T15:44:16.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faafa400700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 0x7faaf419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf9000700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 0x7faaf419fd30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faae00098e0 tx=0x7faae00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf2000700 1 -- 172.21.15.90:0/1167755602 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0027070 con 0x7faaf4104ff0 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf2000700 1 -- 172.21.15.90:0/1167755602 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae0022470 con 0x7faaf4104ff0 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.682+0000 7faaf2000700 1 -- 172.21.15.90:0/1167755602 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0017650 con 0x7faaf4104ff0 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.683+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419a3a0 con 0x7faaf4104ff0 2024-06-02T15:44:16.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.683+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf4102260 con 0x7faaf4104ff0 2024-06-02T15:44:16.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.687+0000 7faaf2000700 1 -- 172.21.15.90:0/1167755602 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faae002a070 con 0x7faaf4104ff0 2024-06-02T15:44:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.689+0000 7faaf2000700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 0x7faadc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.689+0000 7faaf2000700 1 -- 172.21.15.90:0/1167755602 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7faae00e3290 con 0x7faaf4104ff0 2024-06-02T15:44:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.689+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 0x7faadc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.690+0000 7fab06d8eb80 1 Processor -- start 2024-06-02T15:44:16.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.690+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 0x7faadc0b4e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faaf419aaf0 tx=0x7faae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.690+0000 7fab06d8eb80 1 -- start start 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486fef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486feff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557486f4de40 con 0x557486ff5940 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557486f3d9e0 con 0x557486ff5020 2024-06-02T15:44:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557486f41a20 con 0x557486fefad0 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7faaf9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.691+0000 7faaf9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53474/0 (socket says 172.21.15.90:53474) 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 -- 172.21.15.90:0/423097188 learned_addr learned my addr 172.21.15.90:0/423097188 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9000700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486fef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faafa400700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486feff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 -- 172.21.15.90:0/423097188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 msgr2=0x557486feff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486feff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 -- 172.21.15.90:0/423097188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 msgr2=0x557486fef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486fef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 -- 172.21.15.90:0/423097188 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557486f06420 con 0x557486ff5020 2024-06-02T15:44:16.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9000700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486fef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:16.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.692+0000 7faaf9a00700 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faae80043c0 tx=0x7faae8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279ff0cecc75f322 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.693+0000 7faae6a00700 1 -- 172.21.15.90:0/423097188 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8003dc0 con 0x557486ff5020 2024-06-02T15:44:16.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.693+0000 7faae6a00700 1 -- 172.21.15.90:0/423097188 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae801e8d0 con 0x557486ff5020 2024-06-02T15:44:16.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.693+0000 7faae6a00700 1 -- 172.21.15.90:0/423097188 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae801d890 con 0x557486ff5020 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 -- 172.21.15.90:0/423097188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 msgr2=0x557486ff5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faae80043c0 tx=0x7faae8014000 comp rx=0 tx=0).stop 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 -- 172.21.15.90:0/423097188 shutdown_connections 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486feff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff5400 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/423097188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486fef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 -- 172.21.15.90:0/423097188 >> 172.21.15.90:0/423097188 conn(0x557486f1dbe0 msgr2=0x557486fea150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.694+0000 7fab06d8eb80 1 -- 172.21.15.90:0/423097188 shutdown_connections 2024-06-02T15:44:16.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.695+0000 7fab06d8eb80 1 -- 172.21.15.90:0/423097188 wait complete. 2024-06-02T15:44:16.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.695+0000 7fab06d8eb80 1 -- start start 2024-06-02T15:44:16.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.696+0000 7fab06d8eb80 1 Processor -- start 2024-06-02T15:44:16.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.696+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486ffb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.696+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486ff8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557486e43fa0 con 0x557486ff5940 2024-06-02T15:44:16.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557486ede760 con 0x557486ff5020 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557486ffc460 con 0x557486fefad0 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7fab06d8eb80 10 client.0 _ll_register_callbacks cb 0x557486ee7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7faaf9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486ffb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7faaf9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486ffb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55970/0 (socket says 172.21.15.90:55970) 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7faaf9a00700 1 -- 172.21.15.90:0/1030959732 learned_addr learned my addr 172.21.15.90:0/1030959732 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7faaf9000700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.697+0000 7faafa400700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486ff8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 -- 172.21.15.90:0/1030959732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 msgr2=0x557486ffb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486ffb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 -- 172.21.15.90:0/1030959732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 msgr2=0x557486ff8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486ff8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 -- 172.21.15.90:0/1030959732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557486e8d8d0 con 0x557486ff5020 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faafa400700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486ff8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faaf9000700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff8a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faae00a77a0 tx=0x7faae00aa430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdfe22f8775f1c00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faadb400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00aa530 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7faae00094b0 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.698+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae801e8d0 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.699+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faae00ae8e0 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.699+0000 7fab06d8eb80 10 client.25313 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.699+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557486f0e060 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.699+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae801e8d0 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.699+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7faae00b2f70 con 0x557486ff5020 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 10 client.25313 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 20 client.25313 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5574870117f0 0x557487013be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 4 client.25313 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557486f4fd90 con 0x5574870117f0 2024-06-02T15:44:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.700+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5574870117f0 0x557487013be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.701+0000 7faaf9a00700 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5574870117f0 0x557487013be0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89562487c79a3e15 server_cookie=75af8680d692a540 in_seq=0 out_seq=0 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.701+0000 7faadb400700 10 client.25313 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.701+0000 7faadb400700 1 -- 172.21.15.90:0/1030959732 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557486f4fd90 con 0x5574870117f0 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.701+0000 7faadb400700 10 client.25313 handle_command_reply: tid=0 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.701+0000 7fab06d8eb80 1 client.25313 shutdown 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 msgr2=0x557486ff8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff8a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7faae00a77a0 tx=0x7faae00aa430 comp rx=0 tx=0).stop 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 shutdown_connections 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5574870117f0 0x557487013be0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5940 0x557486ff8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7faadb400700 0 client.25313 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486fefad0 0x557486ffb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/1030959732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5020 0x557486ff8a50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 >> 172.21.15.90:0/1030959732 conn(0x557486f1dbe0 msgr2=0x557486fee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 shutdown_connections 2024-06-02T15:44:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 1 -- 172.21.15.90:0/1030959732 wait complete. 2024-06-02T15:44:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.702+0000 7fab06d8eb80 20 client.25313 trim_cache size 0 max 16384 2024-06-02T15:44:16.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 Processor -- start 2024-06-02T15:44:16.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 -- start start 2024-06-02T15:44:16.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 0x557486fe9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5940 0x557486ff8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41af870 con 0x557486ff5020 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ad100 con 0x557486fefad0 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.735+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41aaf50 con 0x557486ff5940 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49626/0 (socket says 172.21.15.90:49626) 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9000700 1 -- 172.21.15.90:0/2054614579 learned_addr learned my addr 172.21.15.90:0/2054614579 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faafa400700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5940 0x557486ff8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 0x557486fe9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 -- 172.21.15.90:0/2054614579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5940 msgr2=0x557486ff8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5940 0x557486ff8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 -- 172.21.15.90:0/2054614579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 msgr2=0x557486fea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9a00700 1 -- 172.21.15.90:0/2054614579 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574870141c0 con 0x557486fefad0 2024-06-02T15:44:16.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.736+0000 7faaf9000700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:16.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.737+0000 7faaf9a00700 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 0x557486fe9c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557487014390 tx=0x7faae8004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bf1d5d014569ca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.737+0000 7faae6000700 1 -- 172.21.15.90:0/2054614579 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8005240 con 0x557486fefad0 2024-06-02T15:44:16.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.737+0000 7faae6000700 1 -- 172.21.15.90:0/2054614579 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae8035030 con 0x557486fefad0 2024-06-02T15:44:16.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.737+0000 7faae6000700 1 -- 172.21.15.90:0/2054614579 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae801d4c0 con 0x557486fefad0 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 -- 172.21.15.90:0/2054614579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 msgr2=0x557486fe9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 0x557486fe9c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x557487014390 tx=0x7faae8004370 comp rx=0 tx=0).stop 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 -- 172.21.15.90:0/2054614579 shutdown_connections 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5940 0x557486ff8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486fefad0 0x557486fe9c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/2054614579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486ff5020 0x557486fea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 -- 172.21.15.90:0/2054614579 >> 172.21.15.90:0/2054614579 conn(0x557486f1dbe0 msgr2=0x557486ff7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 -- 172.21.15.90:0/2054614579 shutdown_connections 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.738+0000 7fab06d8eb80 1 -- 172.21.15.90:0/2054614579 wait complete. 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.739+0000 7fab06d8eb80 1 -- start start 2024-06-02T15:44:16.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.739+0000 7fab06d8eb80 1 Processor -- start 2024-06-02T15:44:16.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486fefad0 0x55748700a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5940 0x557487007490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41ab5c0 con 0x557486fefad0 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41aca30 con 0x557486ff5940 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41abc90 con 0x557486ff5020 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7fab06d8eb80 10 client.0 _ll_register_callbacks cb 0x557486ee7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55980/0 (socket says 172.21.15.90:55980) 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 -- 172.21.15.90:0/3474258576 learned_addr learned my addr 172.21.15.90:0/3474258576 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9a00700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486fefad0 0x55748700a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faafa400700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5940 0x557487007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 -- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5940 msgr2=0x557487007490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5940 0x557487007490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 -- 172.21.15.90:0/3474258576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486fefad0 msgr2=0x55748700a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486fefad0 0x55748700a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.740+0000 7faaf9000700 1 -- 172.21.15.90:0/3474258576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557487007a00 con 0x557486ff5020 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7faaf9000700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faae00aa4c0 tx=0x7faae0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e7b6addd7691dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:16.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7faadb400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00b3300 con 0x557486ff5020 2024-06-02T15:44:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7faae00051f0 con 0x557486ff5020 2024-06-02T15:44:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae8035030 con 0x557486ff5020 2024-06-02T15:44:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faae00053f0 con 0x557486ff5020 2024-06-02T15:44:16.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7fab06d8eb80 10 client.25291 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.741+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55748707d0a0 con 0x557486ff5020 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae8035030 con 0x557486ff5020 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7faae00a5ea0 con 0x557486ff5020 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 10 client.25291 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 20 client.25291 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55748707d510 0x55748707f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 4 client.25291 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557486f49570 con 0x55748707d510 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.742+0000 7faaf9a00700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55748707d510 0x55748707f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.743+0000 7faadb400700 10 client.25291 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:16.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.743+0000 7faaf9a00700 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55748707d510 0x55748707f900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad46d29754634db server_cookie=116643845c1e2f4 in_seq=0 out_seq=0 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.743+0000 7faadb400700 1 -- 172.21.15.90:0/3474258576 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557486f49570 con 0x55748707d510 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.743+0000 7faadb400700 10 client.25291 handle_command_reply: tid=0 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.743+0000 7fab06d8eb80 1 client.25291 shutdown 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 msgr2=0x557487006f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faae00aa4c0 tx=0x7faae0005e80 comp rx=0 tx=0).stop 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 shutdown_connections 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55748707d510 0x55748707f900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557486ff5940 0x557487007490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557486fefad0 0x55748700a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 --2- 172.21.15.90:0/3474258576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557486ff5020 0x557487006f50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 >> 172.21.15.90:0/3474258576 conn(0x557486f1dbe0 msgr2=0x557486ff7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7faadb400700 0 client.25291 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 shutdown_connections 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 1 -- 172.21.15.90:0/3474258576 wait complete. 2024-06-02T15:44:16.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.744+0000 7fab06d8eb80 20 client.25291 trim_cache size 0 max 16384 2024-06-02T15:44:16.745 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:44:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:44:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:44:16.745 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.745+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 msgr2=0x7faadc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.745+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 0x7faadc0b4e40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faaf419aaf0 tx=0x7faae8009260 comp rx=0 tx=0).stop 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.745+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 msgr2=0x7faaf419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.745+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 0x7faaf419fd30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faae00098e0 tx=0x7faae00057c0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 shutdown_connections 2024-06-02T15:44:16.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faadc0b2980 0x7faadc0b4e40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf410dab0 0x7faaf41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faaf41046d0 0x7faaf419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 --2- 172.21.15.90:0/1167755602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faaf4104ff0 0x7faaf419fd30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 >> 172.21.15.90:0/1167755602 conn(0x7faaf40fde60 msgr2=0x7faaf40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 shutdown_connections 2024-06-02T15:44:16.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:16.746+0000 7faafae00700 1 -- 172.21.15.90:0/1167755602 wait complete. 2024-06-02T15:44:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:17 smithi090 ceph-mon[31595]: pgmap v509: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-02T15:44:17.732 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:44:17.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:44:17.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:17 smithi120 ceph-mon[36451]: pgmap v509: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-02T15:44:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:17 smithi135 ceph-mon[36839]: pgmap v509: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-06-02T15:44:18.180 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:19 smithi090 ceph-mon[31595]: pgmap v510: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-02T15:44:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:19 smithi120 ceph-mon[36451]: pgmap v510: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-02T15:44:19.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:19 smithi135 ceph-mon[36839]: pgmap v510: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 -- 172.21.15.90:0/3906157323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c105650 msgr2=0x7f428c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 --2- 172.21.15.90:0/3906157323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c105650 0x7f428c105ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4278009990 tx=0x7f4278009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 -- 172.21.15.90:0/3906157323 shutdown_connections 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 --2- 172.21.15.90:0/3906157323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c106010 0x7f428c112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 --2- 172.21.15.90:0/3906157323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c10b650 0x7f428c10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 --2- 172.21.15.90:0/3906157323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c105650 0x7f428c105ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.549+0000 7f428be00700 1 -- 172.21.15.90:0/3906157323 >> 172.21.15.90:0/3906157323 conn(0x7f428c101120 msgr2=0x7f428c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:20.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.550+0000 7f428be00700 1 -- 172.21.15.90:0/3906157323 shutdown_connections 2024-06-02T15:44:20.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.550+0000 7f428be00700 1 -- 172.21.15.90:0/3906157323 wait complete. 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.551+0000 7f428be00700 1 Processor -- start 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.551+0000 7f428be00700 1 -- start start 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.552+0000 7f428be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.552+0000 7f428be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 0x7f428c10f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.552+0000 7f428be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 0x7f428c10fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.552+0000 7f428be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1158e0 con 0x7f428c10b650 2024-06-02T15:44:20.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428be00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f428c115760 con 0x7f428c105650 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53512/0 (socket says 172.21.15.90:53512) 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428aa00700 1 -- 172.21.15.90:0/1074210668 learned_addr learned my addr 172.21.15.90:0/1074210668 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f428c115a60 con 0x7f428c106010 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428b400700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 0x7f428c10fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.553+0000 7f428a000700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 0x7f428c10f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 -- 172.21.15.90:0/1074210668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 msgr2=0x7f428c10f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:20.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 0x7f428c10f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 -- 172.21.15.90:0/1074210668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 msgr2=0x7f428c1123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c1a1da0 con 0x7f428c10b650 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428aa00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428a000700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 0x7f428c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.554+0000 7f428b400700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 0x7f428c10fa10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f428400efc0 tx=0x7f428400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.555+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840092e0 con 0x7f428c10b650 2024-06-02T15:44:20.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.555+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4284004bb0 con 0x7f428c10b650 2024-06-02T15:44:20.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.555+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284004020 con 0x7f428c10b650 2024-06-02T15:44:20.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.555+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c1a2030 con 0x7f428c10b650 2024-06-02T15:44:20.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.555+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a2520 con 0x7f428c10b650 2024-06-02T15:44:20.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.556+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c106d90 con 0x7f428c10b650 2024-06-02T15:44:20.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.559+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f428401b030 con 0x7f428c10b650 2024-06-02T15:44:20.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.561+0000 7f4283400700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 0x7f42700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:20.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.561+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f42840e2670 con 0x7f428c10b650 2024-06-02T15:44:20.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.562+0000 7f428aa00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 0x7f42700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:20.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.563+0000 7f428aa00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 0x7f42700b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f427c011550 tx=0x7f427c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:20.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.568+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4284022060 con 0x7f428c10b650 2024-06-02T15:44:20.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.945+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f428c110990 con 0x7f428c10b650 2024-06-02T15:44:20.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.946+0000 7f4283400700 1 -- 172.21.15.90:0/1074210668 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f4284010040 con 0x7f428c10b650 2024-06-02T15:44:20.943 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:44:20.943 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:43:15.667079+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 msgr2=0x7f42700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 0x7f42700b4dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f427c011550 tx=0x7f427c009260 comp rx=0 tx=0).stop 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 msgr2=0x7f428c10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 0x7f428c10fa10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f428400efc0 tx=0x7f428400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 shutdown_connections 2024-06-02T15:44:20.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42700b2900 0x7f42700b4dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f428c10b650 0x7f428c10fa10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f428c105650 0x7f428c1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 --2- 172.21.15.90:0/1074210668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f428c106010 0x7f428c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 >> 172.21.15.90:0/1074210668 conn(0x7f428c101120 msgr2=0x7f428c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 shutdown_connections 2024-06-02T15:44:20.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:20.950+0000 7f428be00700 1 -- 172.21.15.90:0/1074210668 wait complete. 2024-06-02T15:44:20.949 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-06-02T15:44:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:21 smithi090 ceph-mon[31595]: pgmap v511: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:21 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1074210668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:21 smithi120 ceph-mon[36451]: pgmap v511: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:21 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1074210668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:21 smithi135 ceph-mon[36839]: pgmap v511: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:21 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1074210668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:21.843 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:44:22.351 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:23.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:23 smithi090 ceph-mon[31595]: pgmap v512: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:44:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:23 smithi120 ceph-mon[36451]: pgmap v512: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:44:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:23 smithi135 ceph-mon[36839]: pgmap v512: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-06-02T15:44:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 -- 172.21.15.90:0/469822161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 msgr2=0x7f96a4106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 --2- 172.21.15.90:0/469822161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a4106fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f968c009990 tx=0x7f968c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 -- 172.21.15.90:0/469822161 shutdown_connections 2024-06-02T15:44:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 --2- 172.21.15.90:0/469822161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 0x7f96a410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 --2- 172.21.15.90:0/469822161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 --2- 172.21.15.90:0/469822161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a4106fd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 -- 172.21.15.90:0/469822161 >> 172.21.15.90:0/469822161 conn(0x7f96a40fde60 msgr2=0x7f96a4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.492+0000 7f96a9800700 1 -- 172.21.15.90:0/469822161 shutdown_connections 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.493+0000 7f96a9800700 1 -- 172.21.15.90:0/469822161 wait complete. 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.494+0000 7f96a9800700 1 Processor -- start 2024-06-02T15:44:24.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.494+0000 7f96a9800700 1 -- start start 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a410b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 0x7f96a410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4111460 con 0x7f96a41025d0 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41112e0 con 0x7f96a4102f80 2024-06-02T15:44:24.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a9800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41115e0 con 0x7f96a41076b0 2024-06-02T15:44:24.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50582/0 (socket says 172.21.15.90:50582) 2024-06-02T15:44:24.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.495+0000 7f96a3e00700 1 -- 172.21.15.90:0/53930260 learned_addr learned my addr 172.21.15.90:0/53930260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:24.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a8e00700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 0x7f96a410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3400700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a410b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3e00700 1 -- 172.21.15.90:0/53930260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 msgr2=0x7f96a410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3e00700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 0x7f96a410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3e00700 1 -- 172.21.15.90:0/53930260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 msgr2=0x7f96a410b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3e00700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a410b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.496+0000 7f96a3e00700 1 -- 172.21.15.90:0/53930260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41ac5e0 con 0x7f96a41025d0 2024-06-02T15:44:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.497+0000 7f96a3e00700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9a0 tx=0x7f969400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.497+0000 7f96a0c00700 1 -- 172.21.15.90:0/53930260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969400cbd0 con 0x7f96a41025d0 2024-06-02T15:44:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.497+0000 7f96a0c00700 1 -- 172.21.15.90:0/53930260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f969400cd70 con 0x7f96a41025d0 2024-06-02T15:44:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.497+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41ac7b0 con 0x7f96a41025d0 2024-06-02T15:44:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.498+0000 7f96a0c00700 1 -- 172.21.15.90:0/53930260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694018680 con 0x7f96a41025d0 2024-06-02T15:44:24.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.498+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41acc60 con 0x7f96a41025d0 2024-06-02T15:44:24.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.502+0000 7f96a0c00700 1 -- 172.21.15.90:0/53930260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9694010430 con 0x7f96a41025d0 2024-06-02T15:44:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.504+0000 7f96a0c00700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 0x7f96880b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.504+0000 7f96a0c00700 1 -- 172.21.15.90:0/53930260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f96940e28c0 con 0x7f96a41025d0 2024-06-02T15:44:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.505+0000 7f96a3400700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 0x7f96880b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.506+0000 7f96a3400700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 0x7f96880b4e90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96a410c980 tx=0x7f968c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.507+0000 7f96b5724b80 1 Processor -- start 2024-06-02T15:44:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 -- start start 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 0x563e188db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563e18838e40 con 0x563e188db390 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e188289e0 con 0x563e188e07c0 2024-06-02T15:44:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96b5724b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e1882ca20 con 0x563e188e10e0 2024-06-02T15:44:24.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58776/0 (socket says 172.21.15.90:58776) 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.508+0000 7f96a3e00700 1 -- 172.21.15.90:0/3624080922 learned_addr learned my addr 172.21.15.90:0/3624080922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 -- 172.21.15.90:0/3624080922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 msgr2=0x563e188dad70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188dad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 -- 172.21.15.90:0/3624080922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 msgr2=0x563e188db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a8e00700 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 0x563e188db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 0x563e188db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 -- 172.21.15.90:0/3624080922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e187f1420 con 0x563e188e07c0 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a8e00700 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 0x563e188db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.509+0000 7f96a3e00700 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96940a9a60 tx=0x7f96940b2370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4614c7f2e48840c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.510+0000 7f9699600700 1 -- 172.21.15.90:0/3624080922 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940a9ad0 con 0x563e188e07c0 2024-06-02T15:44:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.510+0000 7f9699600700 1 -- 172.21.15.90:0/3624080922 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96940ade80 con 0x563e188e07c0 2024-06-02T15:44:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.510+0000 7f9699600700 1 -- 172.21.15.90:0/3624080922 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940b2700 con 0x563e188e07c0 2024-06-02T15:44:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 -- 172.21.15.90:0/3624080922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 msgr2=0x563e188e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96940a9a60 tx=0x7f96940b2370 comp rx=0 tx=0).stop 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 -- 172.21.15.90:0/3624080922 shutdown_connections 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188db390 0x563e188db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188e07c0 0x563e188e0ba0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3624080922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 -- 172.21.15.90:0/3624080922 >> 172.21.15.90:0/3624080922 conn(0x563e18808be0 msgr2=0x563e188d5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 -- 172.21.15.90:0/3624080922 shutdown_connections 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.511+0000 7f96b5724b80 1 -- 172.21.15.90:0/3624080922 wait complete. 2024-06-02T15:44:24.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.512+0000 7f96b5724b80 1 -- start start 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.512+0000 7f96b5724b80 1 Processor -- start 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563e1872efa0 con 0x563e188e10e0 2024-06-02T15:44:24.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563e187c9760 con 0x563e188db390 2024-06-02T15:44:24.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e188e7c00 con 0x563e188e07c0 2024-06-02T15:44:24.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58782/0 (socket says 172.21.15.90:58782) 2024-06-02T15:44:24.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96a3e00700 1 -- 172.21.15.90:0/847404027 learned_addr learned my addr 172.21.15.90:0/847404027 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:24.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.513+0000 7f96b5724b80 10 client.0 _ll_register_callbacks cb 0x563e187d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 -- 172.21.15.90:0/847404027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 msgr2=0x563e188e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3400700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a8e00700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 -- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 msgr2=0x563e188e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 -- 172.21.15.90:0/847404027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e187788d0 con 0x563e188db390 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3400700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a8e00700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:24.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96a3e00700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96940a5b50 tx=0x7f96940aa310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c4d76ad323c1d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f9692000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940c2070 con 0x563e188db390 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f96940aae00 con 0x563e188db390 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.514+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96940ade80 con 0x563e188db390 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.515+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96940ba050 con 0x563e188db390 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.515+0000 7f96b5724b80 10 client.25337 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.515+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e187f9060 con 0x563e188db390 2024-06-02T15:44:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.515+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96940ade80 con 0x563e188db390 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f96940b4c30 con 0x563e188db390 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 10 client.25337 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 10 client.25337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 20 client.25337 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e188f45a0 0x563e188f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 4 client.25337 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.516+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1883ad90 con 0x563e188f45a0 2024-06-02T15:44:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.517+0000 7f96a3400700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e188f45a0 0x563e188f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.517+0000 7f96a3400700 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e188f45a0 0x563e188f6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b0c49565b15365f server_cookie=1056981888dcd2fe in_seq=0 out_seq=0 2024-06-02T15:44:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.517+0000 7f9692000700 10 client.25337 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:24.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.518+0000 7f9692000700 1 -- 172.21.15.90:0/847404027 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563e1883ad90 con 0x563e188f45a0 2024-06-02T15:44:24.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.518+0000 7f9692000700 10 client.25337 handle_command_reply: tid=0 2024-06-02T15:44:24.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.518+0000 7f96b5724b80 1 client.25337 shutdown 2024-06-02T15:44:24.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 msgr2=0x563e188e6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96940a5b50 tx=0x7f96940aa310 comp rx=0 tx=0).stop 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 shutdown_connections 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e188f45a0 0x563e188f6990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188e6e80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 --2- 172.21.15.90:0/847404027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f9692000700 0 client.25337 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.519+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 >> 172.21.15.90:0/847404027 conn(0x563e18808be0 msgr2=0x563e188d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.520+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 shutdown_connections 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.520+0000 7f96b5724b80 1 -- 172.21.15.90:0/847404027 wait complete. 2024-06-02T15:44:24.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.520+0000 7f96b5724b80 20 client.25337 trim_cache size 0 max 16384 2024-06-02T15:44:24.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.536+0000 7f96b5724b80 1 Processor -- start 2024-06-02T15:44:24.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.536+0000 7f96b5724b80 1 -- start start 2024-06-02T15:44:24.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.536+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188dad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58790/0 (socket says 172.21.15.90:58790) 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:48568/0 (socket says 172.21.15.90:48568) 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96b5724b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b80f0 con 0x563e188e10e0 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b5980 con 0x563e188db390 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a3e00700 1 -- 172.21.15.90:0/3401372879 learned_addr learned my addr 172.21.15.90:0/3401372879 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b37d0 con 0x563e188e07c0 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.537+0000 7f96a8e00700 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 -- 172.21.15.90:0/3401372879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 msgr2=0x563e188dad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188dad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 -- 172.21.15.90:0/3401372879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 msgr2=0x563e188d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 -- 172.21.15.90:0/3401372879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e188e2790 con 0x563e188e07c0 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f96a3400700 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f968c0098e0 tx=0x7f968c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef08a8e086ef79b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f9698c00700 1 -- 172.21.15.90:0/3401372879 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c035070 con 0x563e188e07c0 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f9698c00700 1 -- 172.21.15.90:0/3401372879 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c004690 con 0x563e188e07c0 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.538+0000 7f9698c00700 1 -- 172.21.15.90:0/3401372879 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c00f680 con 0x563e188e07c0 2024-06-02T15:44:24.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 msgr2=0x563e188e4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f968c0098e0 tx=0x7f968c01e000 comp rx=0 tx=0).stop 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 shutdown_connections 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e10e0 0x563e188d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 --2- 172.21.15.90:0/3401372879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e07c0 0x563e188e4890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.539+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 >> 172.21.15.90:0/3401372879 conn(0x563e18808be0 msgr2=0x563e188e14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.543+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 shutdown_connections 2024-06-02T15:44:24.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.543+0000 7f96b5724b80 1 -- 172.21.15.90:0/3401372879 wait complete. 2024-06-02T15:44:24.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.544+0000 7f96b5724b80 1 -- start start 2024-06-02T15:44:24.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.544+0000 7f96b5724b80 1 Processor -- start 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e07c0 0x563e188f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b3e40 con 0x563e188e07c0 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b52b0 con 0x563e188db390 2024-06-02T15:44:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b4510 con 0x563e188e10e0 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96b5724b80 10 client.0 _ll_register_callbacks cb 0x563e187d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96a3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58794/0 (socket says 172.21.15.90:58794) 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96a3e00700 1 -- 172.21.15.90:0/1258832477 learned_addr learned my addr 172.21.15.90:0/1258832477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.545+0000 7f96a8e00700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 -- 172.21.15.90:0/1258832477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 msgr2=0x563e188f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 -- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e07c0 msgr2=0x563e188f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e07c0 0x563e188f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 -- 172.21.15.90:0/1258832477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e188f7c30 con 0x563e188db390 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a8e00700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f96a3e00700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96940b2400 tx=0x7f96940a9960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f12fbd61278994 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f9692000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940c2040 con 0x563e188db390 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.546+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f96940ca040 con 0x563e188db390 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.547+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968c004690 con 0x563e188db390 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.547+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96940a6110 con 0x563e188db390 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.547+0000 7f96b5724b80 10 client.25343 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.547+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e188f7c30 con 0x563e188db390 2024-06-02T15:44:24.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.547+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c004690 con 0x563e188db390 2024-06-02T15:44:24.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f96940b4c30 con 0x563e188db390 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 10 client.25343 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 10 client.25343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 20 client.25343 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e1895eee0 0x563e18961320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 4 client.25343 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e18834570 con 0x563e1895eee0 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.548+0000 7f96a3400700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e1895eee0 0x563e18961320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.549+0000 7f9692000700 10 client.25343 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.549+0000 7f96a3400700 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e1895eee0 0x563e18961320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0d932c723c83b4c server_cookie=60aff7d37eb0890f in_seq=0 out_seq=0 2024-06-02T15:44:24.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.549+0000 7f9692000700 1 -- 172.21.15.90:0/1258832477 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e18834570 con 0x563e1895eee0 2024-06-02T15:44:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.549+0000 7f9692000700 10 client.25343 handle_command_reply: tid=0 2024-06-02T15:44:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.549+0000 7f96b5724b80 1 client.25343 shutdown 2024-06-02T15:44:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 msgr2=0x563e188f9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.550 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:44:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96940b2400 tx=0x7f96940a9960 comp rx=0 tx=0).stop 2024-06-02T15:44:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 shutdown_connections 2024-06-02T15:44:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x563e1895eee0 0x563e18961320 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e188e10e0 0x563e188f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563e188db390 0x563e188f9de0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f9692000700 0 client.25343 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 --2- 172.21.15.90:0/1258832477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563e188e07c0 0x563e188f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 >> 172.21.15.90:0/1258832477 conn(0x563e18808be0 msgr2=0x563e188e1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.550+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 shutdown_connections 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96b5724b80 1 -- 172.21.15.90:0/1258832477 wait complete. 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96b5724b80 20 client.25343 trim_cache size 0 max 16384 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 msgr2=0x7f96880b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 0x7f96880b4e90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96a410c980 tx=0x7f968c01a040 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 msgr2=0x7f96a410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.551+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f969400e9a0 tx=0x7f969400ecb0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 shutdown_connections 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96880b29d0 0x7f96880b4e90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41076b0 0x7f96a410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96a41025d0 0x7f96a410f120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 --2- 172.21.15.90:0/53930260 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96a4102f80 0x7f96a410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 >> 172.21.15.90:0/53930260 conn(0x7f96a40fde60 msgr2=0x7f96a40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 shutdown_connections 2024-06-02T15:44:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:24.552+0000 7f96a9800700 1 -- 172.21.15.90:0/53930260 wait complete. 2024-06-02T15:44:24.553 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:24 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:44:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:24 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:44:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:24 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:44:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 -- 172.21.15.90:0/1988707105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 msgr2=0x7f21240758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 --2- 172.21.15.90:0/1988707105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f21240758b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2110009900 tx=0x7f211002a480 comp rx=0 tx=0).stop 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 -- 172.21.15.90:0/1988707105 shutdown_connections 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 --2- 172.21.15.90:0/1988707105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f212410a940 0x7f212410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 --2- 172.21.15.90:0/1988707105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f21240758b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 --2- 172.21.15.90:0/1988707105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2124076a30 0x7f2124074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 -- 172.21.15.90:0/1988707105 >> 172.21.15.90:0/1988707105 conn(0x7f21240fe040 msgr2=0x7f2124100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 -- 172.21.15.90:0/1988707105 shutdown_connections 2024-06-02T15:44:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.058+0000 7f212c000640 1 -- 172.21.15.90:0/1988707105 wait complete. 2024-06-02T15:44:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.060+0000 7f212c000640 1 Processor -- start 2024-06-02T15:44:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.060+0000 7f212c000640 1 -- start start 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f212419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f212410a940 0x7f21241a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2124110830 con 0x7f2124075450 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21241106b0 con 0x7f2124076a30 2024-06-02T15:44:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212c000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21241109b0 con 0x7f212410a940 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f2129600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f212419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f2128c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f2128c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58802/0 (socket says 172.21.15.90:58802) 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f2128c00640 1 -- 172.21.15.90:0/529574330 learned_addr learned my addr 172.21.15.90:0/529574330 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.061+0000 7f212a000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f212410a940 0x7f21241a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 -- 172.21.15.90:0/529574330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f212410a940 msgr2=0x7f21241a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f212410a940 0x7f21241a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 -- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 msgr2=0x7f212419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f212419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 -- 172.21.15.90:0/529574330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21241a2a80 con 0x7f2124076a30 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2128c00640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f211002aa70 tx=0x7f2110003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2122000640 1 -- 172.21.15.90:0/529574330 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211003f070 con 0x7f2124076a30 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2122000640 1 -- 172.21.15.90:0/529574330 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2110031c50 con 0x7f2124076a30 2024-06-02T15:44:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.062+0000 7f2122000640 1 -- 172.21.15.90:0/529574330 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110035030 con 0x7f2124076a30 2024-06-02T15:44:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.063+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a2d10 con 0x7f2124076a30 2024-06-02T15:44:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.063+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a3180 con 0x7f2124076a30 2024-06-02T15:44:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.068+0000 7f212efb7740 1 Processor -- start 2024-06-02T15:44:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.068+0000 7f212efb7740 1 -- start start 2024-06-02T15:44:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43340 0x55f041f43760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f2122000640 1 -- 172.21.15.90:0/529574330 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2110042070 con 0x7f2124076a30 2024-06-02T15:44:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f041eaab90 con 0x55f041f43340 2024-06-02T15:44:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f041e9a730 con 0x55f041f43d30 2024-06-02T15:44:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.069+0000 7f212efb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f041e9e770 con 0x55f041f48ea0 2024-06-02T15:44:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58810/0 (socket says 172.21.15.90:58810) 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 -- 172.21.15.90:0/3144388875 learned_addr learned my addr 172.21.15.90:0/3144388875 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 -- 172.21.15.90:0/3144388875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 msgr2=0x55f041f4b330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 -- 172.21.15.90:0/3144388875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43340 msgr2=0x55f041f43760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43340 0x55f041f43760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 -- 172.21.15.90:0/3144388875 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f041e338b0 con 0x55f041f43d30 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.070+0000 7f2129600640 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2118009d20 tx=0x7f2118011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515ac6b0f7eb5c36 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.071+0000 7f2116600640 1 -- 172.21.15.90:0/3144388875 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211801e070 con 0x55f041f43d30 2024-06-02T15:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.071+0000 7f2116600640 1 -- 172.21.15.90:0/3144388875 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2118019ae0 con 0x55f041f43d30 2024-06-02T15:44:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.071+0000 7f2116600640 1 -- 172.21.15.90:0/3144388875 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211800eac0 con 0x55f041f43d30 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f2122000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 0x7f210c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 -- 172.21.15.90:0/3144388875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 msgr2=0x55f041f48830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2118009d20 tx=0x7f2118011810 comp rx=0 tx=0).stop 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f2122000640 1 -- 172.21.15.90:0/529574330 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f21101027c0 con 0x7f2124076a30 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 -- 172.21.15.90:0/3144388875 shutdown_connections 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f48830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 --2- 172.21.15.90:0/3144388875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43340 0x55f041f43760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 -- 172.21.15.90:0/3144388875 >> 172.21.15.90:0/3144388875 conn(0x55f041f361a0 msgr2=0x55f041f365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212a000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 0x7f210c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 -- 172.21.15.90:0/3144388875 shutdown_connections 2024-06-02T15:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.072+0000 7f212efb7740 1 -- 172.21.15.90:0/3144388875 wait complete. 2024-06-02T15:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.073+0000 7f212efb7740 1 -- start start 2024-06-02T15:44:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.073+0000 7f212efb7740 1 Processor -- start 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f54730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f041b80790 con 0x55f041f43d30 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f041e42bc0 con 0x55f041f43340 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f041f4be50 con 0x55f041f48ea0 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212a000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 0x7f210c0b4eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f211c0099a0 tx=0x7f211c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212efb7740 10 client.0 _ll_register_callbacks cb 0x55f041e20400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58816/0 (socket says 172.21.15.90:58816) 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f2129600640 1 -- 172.21.15.90:0/1051499082 learned_addr learned my addr 172.21.15.90:0/1051499082 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f212a000640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f54730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.074+0000 7f2128c00640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 -- 172.21.15.90:0/1051499082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 msgr2=0x55f041f54730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f54730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 -- 172.21.15.90:0/1051499082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 msgr2=0x55f041f4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 -- 172.21.15.90:0/1051499082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f041baf210 con 0x55f041f43340 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2128c00640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f2129600640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2118005e00 tx=0x7f21180098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406700d9d336d407 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f210aa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:25.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.075+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118026020 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2118004440 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2118019ae0 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2118031050 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f212efb7740 10 client.25355 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f041c7b620 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.076+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2118019ae0 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.077+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f211800e800 con 0x55f041f43340 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.077+0000 7f212efb7740 10 client.25355 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.078+0000 7f212efb7740 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.078+0000 7f212efb7740 20 client.25355 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.078+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041f58a30 0x55f041f5ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.078+0000 7f212efb7740 4 client.25355 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.078+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f041eacae0 con 0x55f041f58a30 2024-06-02T15:44:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.079+0000 7f2128c00640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041f58a30 0x55f041f5ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.080+0000 7f210aa00640 10 client.25355 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.080+0000 7f2128c00640 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041f58a30 0x55f041f5ae20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f1e3165f22efd3 server_cookie=cc86c48395877acb in_seq=0 out_seq=0 2024-06-02T15:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f210aa00640 1 -- 172.21.15.90:0/1051499082 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f041eacae0 con 0x55f041f58a30 2024-06-02T15:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f210aa00640 10 client.25355 handle_command_reply: tid=0 2024-06-02T15:44:25.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 client.25355 shutdown 2024-06-02T15:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 msgr2=0x55f041f4d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2118005e00 tx=0x7f21180098f0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 shutdown_connections 2024-06-02T15:44:25.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041f58a30 0x55f041f5ae20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f210aa00640 0 client.25355 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f54730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 --2- 172.21.15.90:0/1051499082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4d220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 >> 172.21.15.90:0/1051499082 conn(0x55f041f361a0 msgr2=0x55f041f35310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 shutdown_connections 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.081+0000 7f212efb7740 1 -- 172.21.15.90:0/1051499082 wait complete. 2024-06-02T15:44:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.082+0000 7f212efb7740 20 client.25355 trim_cache size 0 max 16384 2024-06-02T15:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.100+0000 7f212efb7740 1 Processor -- start 2024-06-02T15:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.100+0000 7f212efb7740 1 -- start start 2024-06-02T15:44:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.100+0000 7f212efb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.100+0000 7f212efb7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f212efb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f212efb7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f21241abc10 con 0x55f041f43d30 2024-06-02T15:44:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f212efb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a94e0 con 0x55f041f43340 2024-06-02T15:44:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f212efb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a7330 con 0x55f041f48ea0 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2128c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58818/0 (socket says 172.21.15.90:58818) 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 -- 172.21.15.90:0/281709094 learned_addr learned my addr 172.21.15.90:0/281709094 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f212a000640 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 -- 172.21.15.90:0/281709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 msgr2=0x55f041f4cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 -- 172.21.15.90:0/281709094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 msgr2=0x55f041f546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.101+0000 7f2129600640 1 -- 172.21.15.90:0/281709094 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f041f346b0 con 0x55f041f43340 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.102+0000 7f2129600640 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2118009800 tx=0x7f2118011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc72918d599cee04 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.102+0000 7f2115c00640 1 -- 172.21.15.90:0/281709094 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118026070 con 0x55f041f43340 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.102+0000 7f2115c00640 1 -- 172.21.15.90:0/281709094 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2118004240 con 0x55f041f43340 2024-06-02T15:44:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.102+0000 7f2115c00640 1 -- 172.21.15.90:0/281709094 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211801e040 con 0x55f041f43340 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 -- 172.21.15.90:0/281709094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 msgr2=0x55f041f4c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2118009800 tx=0x7f2118011e40 comp rx=0 tx=0).stop 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 -- 172.21.15.90:0/281709094 shutdown_connections 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f48ea0 0x55f041f4cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f43d30 0x55f041f546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 --2- 172.21.15.90:0/281709094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43340 0x55f041f4c840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 -- 172.21.15.90:0/281709094 >> 172.21.15.90:0/281709094 conn(0x55f041f361a0 msgr2=0x55f041f35310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.103+0000 7f212efb7740 1 -- 172.21.15.90:0/281709094 shutdown_connections 2024-06-02T15:44:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.104+0000 7f212efb7740 1 -- 172.21.15.90:0/281709094 wait complete. 2024-06-02T15:44:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.104+0000 7f212efb7740 1 -- start start 2024-06-02T15:44:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.104+0000 7f212efb7740 1 Processor -- start 2024-06-02T15:44:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 0x55f041f51380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 0x55f041fc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a79a0 con 0x55f041f48ea0 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a8e10 con 0x55f041f43d30 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f21241a8070 con 0x55f041f43340 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212efb7740 10 client.0 _ll_register_callbacks cb 0x7f211801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f2128c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f2128c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58820/0 (socket says 172.21.15.90:58820) 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f2128c00640 1 -- 172.21.15.90:0/4255315261 learned_addr learned my addr 172.21.15.90:0/4255315261 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f2129600640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 0x55f041f51380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.105+0000 7f212a000640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 0x55f041fc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 -- 172.21.15.90:0/4255315261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 msgr2=0x55f041f51380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 0x55f041f51380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 -- 172.21.15.90:0/4255315261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 msgr2=0x55f041fc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 0x55f041fc1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 -- 172.21.15.90:0/4255315261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f041f54990 con 0x55f041f43d30 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2129600640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 0x55f041f51380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f212a000640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 0x55f041fc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f2128c00640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21100c6c40 tx=0x7f2110003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbffe6604af82aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f210aa00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.106+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100d2170 con 0x55f041f43d30 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f21100c5c60 con 0x55f041f43d30 2024-06-02T15:44:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2118004240 con 0x55f041f43d30 2024-06-02T15:44:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21100c94e0 con 0x55f041f43d30 2024-06-02T15:44:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f212efb7740 10 client.25361 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f041fc2b60 con 0x55f041f43d30 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21100ca0d0 con 0x55f041f43d30 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.107+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f21100c9c30 con 0x55f041f43d30 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 10 client.25361 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 20 client.25361 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041fc34e0 0x55f041fc5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 4 client.25361 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2118004240 con 0x55f041fc34e0 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f2129600640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041fc34e0 0x55f041fc5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.108+0000 7f2129600640 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041fc34e0 0x55f041fc5900 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3795d1a3bf9a94 server_cookie=32665df773ad870a in_seq=0 out_seq=0 2024-06-02T15:44:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.109+0000 7f210aa00640 10 client.25361 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.111+0000 7f210aa00640 1 -- 172.21.15.90:0/4255315261 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f2118004240 con 0x55f041fc34e0 2024-06-02T15:44:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.111+0000 7f210aa00640 10 client.25361 handle_command_reply: tid=0 2024-06-02T15:44:25.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.111+0000 7f212efb7740 1 client.25361 shutdown 2024-06-02T15:44:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 msgr2=0x55f041f518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21100c6c40 tx=0x7f2110003f90 comp rx=0 tx=0).stop 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 shutdown_connections 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f041fc34e0 0x55f041fc5900 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f041f48ea0 0x55f041fc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f041f43d30 0x55f041f518c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 --2- 172.21.15.90:0/4255315261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f041f43340 0x55f041f51380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 >> 172.21.15.90:0/4255315261 conn(0x55f041f361a0 msgr2=0x55f041f38e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f210aa00640 0 client.25361 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 shutdown_connections 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.112+0000 7f212efb7740 1 -- 172.21.15.90:0/4255315261 wait complete. 2024-06-02T15:44:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.113+0000 7f212efb7740 20 client.25361 trim_cache size 0 max 16384 2024-06-02T15:44:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:44:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:44:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:44:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.113+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 msgr2=0x7f210c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.113+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 0x7f210c0b4eb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f211c0099a0 tx=0x7f211c004a40 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.113+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 msgr2=0x7f212419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.113+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f211002aa70 tx=0x7f2110003b90 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 shutdown_connections 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f210c0b29f0 0x7f210c0b4eb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f212410a940 0x7f21241a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2124076a30 0x7f212419b3a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 --2- 172.21.15.90:0/529574330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2124075450 0x7f212419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 >> 172.21.15.90:0/529574330 conn(0x7f21240fe040 msgr2=0x7f21241068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 shutdown_connections 2024-06-02T15:44:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:25.114+0000 7f212c000640 1 -- 172.21.15.90:0/529574330 wait complete. 2024-06-02T15:44:25.474 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:44:25.708 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:25 smithi120 ceph-mon[36451]: pgmap v513: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:25.709 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:44:25.709 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:44:25.709 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:25 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:44:25.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:25 smithi135 ceph-mon[36839]: pgmap v513: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:25.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:44:25.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:44:25.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:25 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:44:25.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:25 smithi090 ceph-mon[31595]: pgmap v513: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:44:25.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:44:25.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:44:25.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:25 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:44:25.924 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:27.553 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:27 smithi090 ceph-mon[31595]: pgmap v514: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-02T15:44:27.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:27 smithi120 ceph-mon[36451]: pgmap v514: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-02T15:44:27.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:27 smithi135 ceph-mon[36839]: pgmap v514: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-06-02T15:44:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.110+0000 7fa584e00700 1 -- 172.21.15.90:0/2268385941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58006b220 msgr2=0x7fa58006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.110+0000 7fa584e00700 1 --2- 172.21.15.90:0/2268385941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58006b220 0x7fa58006b6a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009990 tx=0x7fa56c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 -- 172.21.15.90:0/2268385941 shutdown_connections 2024-06-02T15:44:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 --2- 172.21.15.90:0/2268385941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 0x7fa5801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 --2- 172.21.15.90:0/2268385941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5801095b0 0x7fa580109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 --2- 172.21.15.90:0/2268385941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa58006b220 0x7fa58006b6a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 -- 172.21.15.90:0/2268385941 >> 172.21.15.90:0/2268385941 conn(0x7fa5800793f0 msgr2=0x7fa580079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 -- 172.21.15.90:0/2268385941 shutdown_connections 2024-06-02T15:44:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.111+0000 7fa584e00700 1 -- 172.21.15.90:0/2268385941 wait complete. 2024-06-02T15:44:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.113+0000 7fa584e00700 1 Processor -- start 2024-06-02T15:44:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.113+0000 7fa584e00700 1 -- start start 2024-06-02T15:44:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.114+0000 7fa584e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.114+0000 7fa584e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 0x7fa58010eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.114+0000 7fa584e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801095b0 0x7fa5801a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa584e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa580113710 con 0x7fa58006b220 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa584e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa580113590 con 0x7fa58006bbe0 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa584e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa580113890 con 0x7fa5801095b0 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa57de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa57de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50714/0 (socket says 172.21.15.90:50714) 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa57de00700 1 -- 172.21.15.90:0/1250169026 learned_addr learned my addr 172.21.15.90:0/1250169026 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa57d400700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 0x7fa58010eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.115+0000 7fa57e800700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801095b0 0x7fa5801a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.116+0000 7fa57de00700 1 -- 172.21.15.90:0/1250169026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801095b0 msgr2=0x7fa5801a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.116+0000 7fa57de00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801095b0 0x7fa5801a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.116+0000 7fa57de00700 1 -- 172.21.15.90:0/1250169026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 msgr2=0x7fa58010eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.116+0000 7fa57de00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 0x7fa58010eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.116+0000 7fa57de00700 1 -- 172.21.15.90:0/1250169026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5801a5220 con 0x7fa58006b220 2024-06-02T15:44:28.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa57de00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa57000e9a0 tx=0x7fa57000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa576a00700 1 -- 172.21.15.90:0/1250169026 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57000cbd0 con 0x7fa58006b220 2024-06-02T15:44:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa576a00700 1 -- 172.21.15.90:0/1250169026 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa57000cd70 con 0x7fa58006b220 2024-06-02T15:44:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa576a00700 1 -- 172.21.15.90:0/1250169026 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570018680 con 0x7fa58006b220 2024-06-02T15:44:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5801a5510 con 0x7fa58006b220 2024-06-02T15:44:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.117+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5801acdc0 con 0x7fa58006b220 2024-06-02T15:44:28.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.124+0000 7fa576a00700 1 -- 172.21.15.90:0/1250169026 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa570010970 con 0x7fa58006b220 2024-06-02T15:44:28.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.125+0000 7fa58f6b5b80 1 Processor -- start 2024-06-02T15:44:28.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.127+0000 7fa58f6b5b80 1 -- start start 2024-06-02T15:44:28.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.127+0000 7fa576a00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 0x7fa5640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.127+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a278333290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.127+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.128+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a27832dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.128+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a27828be40 con 0x55a27832d880 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.128+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a27827b9e0 con 0x55a2783337d0 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.128+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a27827fa20 con 0x55a278332eb0 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.128+0000 7fa576a00700 1 -- 172.21.15.90:0/1250169026 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa5700e2880 con 0x7fa58006b220 2024-06-02T15:44:28.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.129+0000 7fa57e800700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 0x7fa5640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.129+0000 7fa57d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.129+0000 7fa57d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58850/0 (socket says 172.21.15.90:58850) 2024-06-02T15:44:28.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.129+0000 7fa57d400700 1 -- 172.21.15.90:0/2016720717 learned_addr learned my addr 172.21.15.90:0/2016720717 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:28.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.130+0000 7fa57e800700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 0x7fa5640b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5780093d0 tx=0x7fa57800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.131+0000 7fa57de00700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a278333290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.131+0000 7fa57e800700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a27832dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.131+0000 7fa57d400700 1 -- 172.21.15.90:0/2016720717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 msgr2=0x55a278333290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.131+0000 7fa57d400700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a278333290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.131+0000 7fa57d400700 1 -- 172.21.15.90:0/2016720717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 msgr2=0x55a27832dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.132+0000 7fa57d400700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a27832dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.132+0000 7fa57d400700 1 -- 172.21.15.90:0/2016720717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a278244420 con 0x55a2783337d0 2024-06-02T15:44:28.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.132+0000 7fa57e800700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a27832dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:28.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.133+0000 7fa57d400700 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa56c015000 tx=0x7fa56c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5749c729fc79e82 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.134+0000 7fa56b400700 1 -- 172.21.15.90:0/2016720717 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00bbb0 con 0x55a2783337d0 2024-06-02T15:44:28.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.134+0000 7fa56b400700 1 -- 172.21.15.90:0/2016720717 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c00bd50 con 0x55a2783337d0 2024-06-02T15:44:28.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.134+0000 7fa56b400700 1 -- 172.21.15.90:0/2016720717 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c003da0 con 0x55a2783337d0 2024-06-02T15:44:28.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2016720717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 msgr2=0x55a27832d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa56c015000 tx=0x7fa56c005f00 comp rx=0 tx=0).stop 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2016720717 shutdown_connections 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a27832dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a278333290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2016720717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a2783337d0 0x55a27832d260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.136+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2016720717 >> 172.21.15.90:0/2016720717 conn(0x55a27825bbe0 msgr2=0x55a278328010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.137+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2016720717 shutdown_connections 2024-06-02T15:44:28.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.137+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2016720717 wait complete. 2024-06-02T15:44:28.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.138+0000 7fa58f6b5b80 1 -- start start 2024-06-02T15:44:28.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.138+0000 7fa58f6b5b80 1 Processor -- start 2024-06-02T15:44:28.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 0x55a278336880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a278336dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a278181fa0 con 0x55a278332eb0 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a27821c760 con 0x55a27832d880 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a27834b190 con 0x55a2783337d0 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.139+0000 7fa58f6b5b80 10 client.0 _ll_register_callbacks cb 0x55a278225c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.140+0000 7fa57de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.140+0000 7fa57de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58866/0 (socket says 172.21.15.90:58866) 2024-06-02T15:44:28.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.140+0000 7fa57de00700 1 -- 172.21.15.90:0/3366629270 learned_addr learned my addr 172.21.15.90:0/3366629270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:28.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.140+0000 7fa57d400700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 0x55a278336880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57e800700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a278336dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57de00700 1 -- 172.21.15.90:0/3366629270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 msgr2=0x55a278336dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57de00700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a278336dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57de00700 1 -- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 msgr2=0x55a278336880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57de00700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 0x55a278336880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.141+0000 7fa57de00700 1 -- 172.21.15.90:0/3366629270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2781cb8d0 con 0x55a27832d880 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.142+0000 7fa57d400700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 0x55a278336880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.142+0000 7fa57de00700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5700a97f0 tx=0x7fa5700b27a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdfc8c9d26f80e19 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.142+0000 7fa55fe00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:28.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.143+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5700a9600 con 0x55a27832d880 2024-06-02T15:44:28.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.143+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa5700a5670 con 0x55a27832d880 2024-06-02T15:44:28.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.143+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c00bd50 con 0x55a27832d880 2024-06-02T15:44:28.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.144+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5700aa1f0 con 0x55a27832d880 2024-06-02T15:44:28.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.144+0000 7fa58f6b5b80 10 client.25373 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:28.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.144+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a27824c060 con 0x55a27832d880 2024-06-02T15:44:28.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c00bd50 con 0x55a27832d880 2024-06-02T15:44:28.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa5700cc0b0 con 0x55a27832d880 2024-06-02T15:44:28.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 10 client.25373 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:28.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 10 client.25373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:44:28.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 20 client.25373 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:28.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a27835a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 4 client.25373 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:28.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.145+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a27828dd90 con 0x55a278357c10 2024-06-02T15:44:28.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.146+0000 7fa57d400700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a27835a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.147+0000 7fa57d400700 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a27835a000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc38ef388ae2e4af server_cookie=5809f08d08f910a9 in_seq=0 out_seq=0 2024-06-02T15:44:28.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.147+0000 7fa55fe00700 10 client.25373 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:28.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.149+0000 7fa55fe00700 1 -- 172.21.15.90:0/3366629270 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a27828dd90 con 0x55a278357c10 2024-06-02T15:44:28.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.149+0000 7fa55fe00700 10 client.25373 handle_command_reply: tid=0 2024-06-02T15:44:28.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.149+0000 7fa58f6b5b80 1 client.25373 shutdown 2024-06-02T15:44:28.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.150+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 msgr2=0x55a2783394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.150+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5700a97f0 tx=0x7fa5700b27a0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 shutdown_connections 2024-06-02T15:44:28.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a278336dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a27835a000 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa55fe00700 0 client.25373 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:28.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a2783394e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/3366629270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a278332eb0 0x55a278336880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.151+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 >> 172.21.15.90:0/3366629270 conn(0x55a27825bbe0 msgr2=0x55a27832c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.152+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 shutdown_connections 2024-06-02T15:44:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.152+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/3366629270 wait complete. 2024-06-02T15:44:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.152+0000 7fa58f6b5b80 20 client.25373 trim_cache size 0 max 16384 2024-06-02T15:44:28.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.183+0000 7fa58f6b5b80 1 Processor -- start 2024-06-02T15:44:28.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 -- start start 2024-06-02T15:44:28.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 0x55a27832dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a27835f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801afa50 con 0x55a27832d880 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5801ad2e0 con 0x55a278332eb0 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5800737e0 con 0x55a2783337d0 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa57de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa57d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 0x55a27832dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.184+0000 7fa57de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50754/0 (socket says 172.21.15.90:50754) 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57de00700 1 -- 172.21.15.90:0/2243201621 learned_addr learned my addr 172.21.15.90:0/2243201621 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57e800700 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a27835f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57d400700 1 -- 172.21.15.90:0/2243201621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 msgr2=0x55a27835f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57d400700 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a27835f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57d400700 1 -- 172.21.15.90:0/2243201621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 msgr2=0x55a2783280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57d400700 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57d400700 1 -- 172.21.15.90:0/2243201621 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a278334e10 con 0x55a278332eb0 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.185+0000 7fa57de00700 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:28.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.186+0000 7fa57d400700 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 0x55a27832dc60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0098e0 tx=0x7fa56c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e3aa8916151448 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.186+0000 7fa56aa00700 1 -- 172.21.15.90:0/2243201621 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0052b0 con 0x55a278332eb0 2024-06-02T15:44:28.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.186+0000 7fa56aa00700 1 -- 172.21.15.90:0/2243201621 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c031070 con 0x55a278332eb0 2024-06-02T15:44:28.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.186+0000 7fa56aa00700 1 -- 172.21.15.90:0/2243201621 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00f6c0 con 0x55a278332eb0 2024-06-02T15:44:28.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2243201621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 msgr2=0x55a27832dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 0x55a27832dc60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0098e0 tx=0x7fa56c000bc0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2243201621 shutdown_connections 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2783337d0 0x55a27835f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a27832d880 0x55a2783280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2243201621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a278332eb0 0x55a27832dc60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2243201621 >> 172.21.15.90:0/2243201621 conn(0x55a27825bbe0 msgr2=0x55a2783279e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2243201621 shutdown_connections 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.187+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2243201621 wait complete. 2024-06-02T15:44:28.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.188+0000 7fa58f6b5b80 1 -- start start 2024-06-02T15:44:28.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.188+0000 7fa58f6b5b80 1 Processor -- start 2024-06-02T15:44:28.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a27835f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 0x55a27835f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa580073e50 con 0x55a2783337d0 2024-06-02T15:44:28.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa5800752c0 con 0x55a27832d880 2024-06-02T15:44:28.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa580074520 con 0x55a278332eb0 2024-06-02T15:44:28.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa57de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa58f6b5b80 10 client.0 _ll_register_callbacks cb 0x55a278225c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa57de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58876/0 (socket says 172.21.15.90:58876) 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa57de00700 1 -- 172.21.15.90:0/2828901361 learned_addr learned my addr 172.21.15.90:0/2828901361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.189+0000 7fa57d400700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a27835f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 -- 172.21.15.90:0/2828901361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 msgr2=0x55a27835f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a27835f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 -- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 msgr2=0x55a27835f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57e800700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 0x55a27835f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 0x55a27835f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 -- 172.21.15.90:0/2828901361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2783c1770 con 0x55a27832d880 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57e800700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 0x55a27835f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57d400700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a27835f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:28.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa57de00700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5700b28c0 tx=0x7fa57000efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8b0cc6ddbf5aae server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.190+0000 7fa55fe00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5700cc2c0 con 0x55a27832d880 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa5700a88e0 con 0x55a27832d880 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa56c031070 con 0x55a27832d880 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa57008d050 con 0x55a27832d880 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa58f6b5b80 10 client.25379 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a2783c28f0 con 0x55a27832d880 2024-06-02T15:44:28.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.191+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56c031070 con 0x55a27832d880 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa5700b2ba0 con 0x55a27832d880 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 10 client.25379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 20 client.25379 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a2783c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:28.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 4 client.25379 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a278287570 con 0x55a278357c10 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.192+0000 7fa57d400700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a2783c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.193+0000 7fa57d400700 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a2783c4fa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e02712f4ca05641 server_cookie=648db6f9959c4d4 in_seq=0 out_seq=0 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.193+0000 7fa55fe00700 10 client.25379 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.194+0000 7fa55fe00700 1 -- 172.21.15.90:0/2828901361 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a278287570 con 0x55a278357c10 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.194+0000 7fa55fe00700 10 client.25379 handle_command_reply: tid=0 2024-06-02T15:44:28.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.194+0000 7fa58f6b5b80 1 client.25379 shutdown 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 msgr2=0x55a278350c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5700b28c0 tx=0x7fa57000efa0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 shutdown_connections 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a2783337d0 0x55a27835f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55a278357c10 0x55a2783c4fa0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a27832d880 0x55a278350c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 --2- 172.21.15.90:0/2828901361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a278332eb0 0x55a27835f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 >> 172.21.15.90:0/2828901361 conn(0x55a27825bbe0 msgr2=0x55a278333c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.195+0000 7fa55fe00700 0 client.25379 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.196+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 shutdown_connections 2024-06-02T15:44:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.196+0000 7fa58f6b5b80 1 -- 172.21.15.90:0/2828901361 wait complete. 2024-06-02T15:44:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.197+0000 7fa58f6b5b80 20 client.25379 trim_cache size 0 max 16384 2024-06-02T15:44:28.194 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 msgr2=0x7fa5640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 0x7fa5640b4dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5780093d0 tx=0x7fa57800b040 comp rx=0 tx=0).stop 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 msgr2=0x7fa58010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa57000e9a0 tx=0x7fa57000ed60 comp rx=0 tx=0).stop 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 shutdown_connections 2024-06-02T15:44:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa5640b2900 0x7fa5640b4dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5801095b0 0x7fa5801a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa58006b220 0x7fa58010e580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 --2- 172.21.15.90:0/1250169026 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa58006bbe0 0x7fa58010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 >> 172.21.15.90:0/1250169026 conn(0x7fa5800793f0 msgr2=0x7fa580102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.198+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 shutdown_connections 2024-06-02T15:44:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:28.199+0000 7fa584e00700 1 -- 172.21.15.90:0/1250169026 wait complete. 2024-06-02T15:44:28.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:28 smithi120 ceph-mon[36451]: pgmap v515: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-02T15:44:28.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:28 smithi135 ceph-mon[36839]: pgmap v515: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-02T15:44:28.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:28 smithi090 ceph-mon[31595]: pgmap v515: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-06-02T15:44:29.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:44:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:31.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:30 smithi090 ceph-mon[31595]: pgmap v516: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:31.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:30 smithi120 ceph-mon[36451]: pgmap v516: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:31.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:30 smithi135 ceph-mon[36839]: pgmap v516: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:31.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 -- 172.21.15.90:0/3782647261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c06b250 msgr2=0x7f658c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:31.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 --2- 172.21.15.90:0/3782647261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c06b250 0x7f658c06b630 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f657c009990 tx=0x7f657c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 -- 172.21.15.90:0/3782647261 shutdown_connections 2024-06-02T15:44:31.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 --2- 172.21.15.90:0/3782647261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c10fff0 0x7f658c1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 --2- 172.21.15.90:0/3782647261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c06b250 0x7f658c06b630 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 --2- 172.21.15.90:0/3782647261 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06bb70 0x7f658c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.937+0000 7f6592600700 1 -- 172.21.15.90:0/3782647261 >> 172.21.15.90:0/3782647261 conn(0x7f658c0793f0 msgr2=0x7f658c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:31.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.938+0000 7f6592600700 1 -- 172.21.15.90:0/3782647261 shutdown_connections 2024-06-02T15:44:31.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.938+0000 7f6592600700 1 -- 172.21.15.90:0/3782647261 wait complete. 2024-06-02T15:44:31.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.940+0000 7f6592600700 1 Processor -- start 2024-06-02T15:44:31.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.940+0000 7f6592600700 1 -- start start 2024-06-02T15:44:31.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.941+0000 7f6592600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.941+0000 7f6592600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 0x7f658c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.941+0000 7f6592600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 0x7f658c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f658b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f658b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:32996/0 (socket says 172.21.15.90:32996) 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f658aa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 0x7f658c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f658c115950 con 0x7f658c10fff0 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f658b400700 1 -- 172.21.15.90:0/2333039993 learned_addr learned my addr 172.21.15.90:0/2333039993 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1157d0 con 0x7f658c06b250 2024-06-02T15:44:31.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f658c115ad0 con 0x7f658c06bb70 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.942+0000 7f658be00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 0x7f658c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 -- 172.21.15.90:0/2333039993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 msgr2=0x7f658c074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 -- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 msgr2=0x7f658c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 0x7f658c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c06ea30 con 0x7f658c06bb70 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658be00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 0x7f658c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658b400700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:31.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.943+0000 7f658aa00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 0x7f658c06deb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6578005950 tx=0x7f65780074a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:31.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.944+0000 7f6583e00700 1 -- 172.21.15.90:0/2333039993 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578013020 con 0x7f658c06bb70 2024-06-02T15:44:31.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.945+0000 7f6583e00700 1 -- 172.21.15.90:0/2333039993 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6578007cc0 con 0x7f658c06bb70 2024-06-02T15:44:31.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.945+0000 7f6583e00700 1 -- 172.21.15.90:0/2333039993 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800eac0 con 0x7f658c06bb70 2024-06-02T15:44:31.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.945+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c072840 con 0x7f658c06bb70 2024-06-02T15:44:31.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.945+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c072d30 con 0x7f658c06bb70 2024-06-02T15:44:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.951+0000 7f6583e00700 1 -- 172.21.15.90:0/2333039993 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f657802c020 con 0x7f658c06bb70 2024-06-02T15:44:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.952+0000 7f659d015b80 1 Processor -- start 2024-06-02T15:44:31.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 -- start start 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af94f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af949ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597af8a7e40 con 0x5597af94eeb0 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5597af8979e0 con 0x5597af949880 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.953+0000 7f659d015b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597af89ba20 con 0x5597af94f7d0 2024-06-02T15:44:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.954+0000 7f658b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.954+0000 7f658b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38356/0 (socket says 172.21.15.90:38356) 2024-06-02T15:44:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.954+0000 7f658b400700 1 -- 172.21.15.90:0/778277021 learned_addr learned my addr 172.21.15.90:0/778277021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.954+0000 7f658be00700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af94f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.954+0000 7f658aa00700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af949ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.955+0000 7f6583e00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 0x7f65700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f6583e00700 1 -- 172.21.15.90:0/2333039993 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f65780f2bc0 con 0x7f658c06bb70 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 -- 172.21.15.90:0/778277021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 msgr2=0x5597af949ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af949ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 -- 172.21.15.90:0/778277021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 msgr2=0x5597af94f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af94f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 -- 172.21.15.90:0/778277021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597af860420 con 0x5597af94f7d0 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658be00700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af94f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:31.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.956+0000 7f658b400700 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f657c005be0 tx=0x7f657c005c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee9bfaf2cabbcc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:31.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.957+0000 7f658be00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 0x7f65700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.958+0000 7f6580c00700 1 -- 172.21.15.90:0/778277021 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c027070 con 0x5597af94f7d0 2024-06-02T15:44:31.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.958+0000 7f6580c00700 1 -- 172.21.15.90:0/778277021 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c003680 con 0x5597af94f7d0 2024-06-02T15:44:31.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.958+0000 7f6580c00700 1 -- 172.21.15.90:0/778277021 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c00f6c0 con 0x5597af94f7d0 2024-06-02T15:44:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.959+0000 7f658be00700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 0x7f65700b4dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f658c06ae10 tx=0x7f6584009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.959+0000 7f659d015b80 1 -- 172.21.15.90:0/778277021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 msgr2=0x5597af949260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.959+0000 7f659d015b80 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f657c005be0 tx=0x7f657c005c10 comp rx=0 tx=0).stop 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.959+0000 7f659d015b80 1 -- 172.21.15.90:0/778277021 shutdown_connections 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.959+0000 7f659d015b80 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af949ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.960+0000 7f659d015b80 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af94f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.960+0000 7f659d015b80 1 --2- 172.21.15.90:0/778277021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af949260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.960+0000 7f659d015b80 1 -- 172.21.15.90:0/778277021 >> 172.21.15.90:0/778277021 conn(0x5597af877be0 msgr2=0x5597af944010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.960+0000 7f659d015b80 1 -- 172.21.15.90:0/778277021 shutdown_connections 2024-06-02T15:44:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.960+0000 7f659d015b80 1 -- 172.21.15.90:0/778277021 wait complete. 2024-06-02T15:44:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.961+0000 7f659d015b80 1 -- start start 2024-06-02T15:44:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.961+0000 7f659d015b80 1 Processor -- start 2024-06-02T15:44:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.962+0000 7f659d015b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af955520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.962+0000 7f659d015b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af9528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.962+0000 7f659d015b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af952e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.963+0000 7f658b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af955520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.963+0000 7f658aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af9528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f658be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af952e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f658be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af952e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38362/0 (socket says 172.21.15.90:38362) 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f658be00700 1 -- 172.21.15.90:0/478758691 learned_addr learned my addr 172.21.15.90:0/478758691 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597af79dfa0 con 0x5597af94eeb0 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5597af838760 con 0x5597af949880 2024-06-02T15:44:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597af9561e0 con 0x5597af94f7d0 2024-06-02T15:44:31.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.964+0000 7f659d015b80 10 client.0 _ll_register_callbacks cb 0x5597af841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.965+0000 7f658aa00700 1 -- 172.21.15.90:0/478758691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 msgr2=0x5597af952e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.965+0000 7f658aa00700 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af952e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.965+0000 7f658aa00700 1 -- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 msgr2=0x5597af955520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.965+0000 7f658aa00700 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af955520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.965+0000 7f658aa00700 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597af7e78d0 con 0x5597af94eeb0 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.966+0000 7f658aa00700 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af9528c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65780f02b0 tx=0x7f65780139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e403cea0dbd997ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.966+0000 7f6575600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.967+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780b9ba0 con 0x5597af94eeb0 2024-06-02T15:44:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.967+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f65780c29b0 con 0x5597af94eeb0 2024-06-02T15:44:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.967+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c003680 con 0x5597af94eeb0 2024-06-02T15:44:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.967+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65780b9540 con 0x5597af94eeb0 2024-06-02T15:44:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.968+0000 7f659d015b80 10 client.15252 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.968+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597af868060 con 0x5597af94eeb0 2024-06-02T15:44:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.968+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c003680 con 0x5597af94eeb0 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f65780b5740 con 0x5597af94eeb0 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 10 client.15252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 20 client.15252 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af962d40 0x5597af965130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 4 client.15252 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.969+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597af8a9d90 con 0x5597af962d40 2024-06-02T15:44:31.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.970+0000 7f658b400700 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af962d40 0x5597af965130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:31.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.970+0000 7f658b400700 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af962d40 0x5597af965130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64686740f305cb07 server_cookie=2a1368841819fa9 in_seq=0 out_seq=0 2024-06-02T15:44:31.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.970+0000 7f6575600700 10 client.15252 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:31.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.971+0000 7f6575600700 1 -- 172.21.15.90:0/478758691 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5597af8a9d90 con 0x5597af962d40 2024-06-02T15:44:31.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.971+0000 7f6575600700 10 client.15252 handle_command_reply: tid=0 2024-06-02T15:44:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.972+0000 7f659d015b80 1 client.15252 shutdown 2024-06-02T15:44:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.972+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 msgr2=0x5597af9528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.972+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af9528c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65780f02b0 tx=0x7f65780139c0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 shutdown_connections 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af962d40 0x5597af965130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af94f7d0 0x5597af952e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af949880 0x5597af955520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 --2- 172.21.15.90:0/478758691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af9528c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 >> 172.21.15.90:0/478758691 conn(0x5597af877be0 msgr2=0x5597af948250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f6575600700 0 client.15252 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:31.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 shutdown_connections 2024-06-02T15:44:31.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.973+0000 7f659d015b80 1 -- 172.21.15.90:0/478758691 wait complete. 2024-06-02T15:44:31.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:31.974+0000 7f659d015b80 20 client.15252 trim_cache size 0 max 16384 2024-06-02T15:44:32.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.003+0000 7f659d015b80 1 Processor -- start 2024-06-02T15:44:32.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.003+0000 7f659d015b80 1 -- start start 2024-06-02T15:44:32.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af949c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 0x5597af9440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b8220 con 0x5597af94eeb0 2024-06-02T15:44:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b5ab0 con 0x5597af94f7d0 2024-06-02T15:44:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.004+0000 7f659d015b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b3980 con 0x5597af949880 2024-06-02T15:44:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.005+0000 7f658b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.005+0000 7f658be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 0x5597af9440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.006+0000 7f658be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 0x5597af9440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33018/0 (socket says 172.21.15.90:33018) 2024-06-02T15:44:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.006+0000 7f658be00700 1 -- 172.21.15.90:0/4283226878 learned_addr learned my addr 172.21.15.90:0/4283226878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.005+0000 7f658b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38368/0 (socket says 172.21.15.90:38368) 2024-06-02T15:44:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.006+0000 7f658aa00700 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af949c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658b400700 1 -- 172.21.15.90:0/4283226878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 msgr2=0x5597af9440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658b400700 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 0x5597af9440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658b400700 1 -- 172.21.15.90:0/4283226878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 msgr2=0x5597af949c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658b400700 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af949c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658b400700 1 -- 172.21.15.90:0/4283226878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597af93c190 con 0x5597af949880 2024-06-02T15:44:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.007+0000 7f658aa00700 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af949c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.008+0000 7f658b400700 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f657c009ce0 tx=0x7f657c019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d115ed50a8d2a82 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:32.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.008+0000 7f6577e00700 1 -- 172.21.15.90:0/4283226878 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c031070 con 0x5597af949880 2024-06-02T15:44:32.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.008+0000 7f6577e00700 1 -- 172.21.15.90:0/4283226878 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c028040 con 0x5597af949880 2024-06-02T15:44:32.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.008+0000 7f6577e00700 1 -- 172.21.15.90:0/4283226878 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c027040 con 0x5597af949880 2024-06-02T15:44:32.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.009+0000 7f659d015b80 1 -- 172.21.15.90:0/4283226878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 msgr2=0x5597af949260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:32.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.009+0000 7f659d015b80 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f657c009ce0 tx=0x7f657c019040 comp rx=0 tx=0).stop 2024-06-02T15:44:32.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 -- 172.21.15.90:0/4283226878 shutdown_connections 2024-06-02T15:44:32.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94f7d0 0x5597af9440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af949260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 --2- 172.21.15.90:0/4283226878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94eeb0 0x5597af949c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 -- 172.21.15.90:0/4283226878 >> 172.21.15.90:0/4283226878 conn(0x5597af877be0 msgr2=0x5597af9439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 -- 172.21.15.90:0/4283226878 shutdown_connections 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.010+0000 7f659d015b80 1 -- 172.21.15.90:0/4283226878 wait complete. 2024-06-02T15:44:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.011+0000 7f659d015b80 1 -- start start 2024-06-02T15:44:32.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.011+0000 7f659d015b80 1 Processor -- start 2024-06-02T15:44:32.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af95b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 0x5597af96a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b3f70 con 0x5597af94f7d0 2024-06-02T15:44:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b53e0 con 0x5597af94eeb0 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1b4640 con 0x5597af949880 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.012+0000 7f659d015b80 10 client.0 _ll_register_callbacks cb 0x5597af841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.013+0000 7f658aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.013+0000 7f658aa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33020/0 (socket says 172.21.15.90:33020) 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.013+0000 7f658aa00700 1 -- 172.21.15.90:0/302309222 learned_addr learned my addr 172.21.15.90:0/302309222 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.013+0000 7f658b400700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af95b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.013+0000 7f658aa00700 1 -- 172.21.15.90:0/302309222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 msgr2=0x5597af95b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.014+0000 7f658be00700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 0x5597af96a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.014+0000 7f658aa00700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af95b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.014+0000 7f658aa00700 1 -- 172.21.15.90:0/302309222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 msgr2=0x5597af96a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:32.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.014+0000 7f658aa00700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 0x5597af96a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.014+0000 7f658aa00700 1 -- 172.21.15.90:0/302309222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597af96a970 con 0x5597af94eeb0 2024-06-02T15:44:32.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.015+0000 7f658b400700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af95b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:32.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.015+0000 7f658be00700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 0x5597af96a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:32.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.016+0000 7f658aa00700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65780b5e50 tx=0x7f65780f2ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666551b27219a660 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:32.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.016+0000 7f6575600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:32.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.016+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780d8060 con 0x5597af94eeb0 2024-06-02T15:44:32.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.017+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f65780b9030 con 0x5597af94eeb0 2024-06-02T15:44:32.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.017+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657c028040 con 0x5597af94eeb0 2024-06-02T15:44:32.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.018+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65780b9720 con 0x5597af94eeb0 2024-06-02T15:44:32.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.018+0000 7f659d015b80 10 client.25397 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:32.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.018+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597af96a970 con 0x5597af94eeb0 2024-06-02T15:44:32.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.018+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f657c028040 con 0x5597af94eeb0 2024-06-02T15:44:32.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f65780131c0 con 0x5597af94eeb0 2024-06-02T15:44:32.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 10 client.25397 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:32.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:32.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 20 client.25397 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:32.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af9cd8a0 0x5597af9cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:32.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 4 client.25397 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:44:32.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597af8a3570 con 0x5597af9cd8a0 2024-06-02T15:44:32.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.020+0000 7f658b400700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af9cd8a0 0x5597af9cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:32.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.021+0000 7f658b400700 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af9cd8a0 0x5597af9cfcc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88fd3ce52f85da3 server_cookie=b8db1e9e9004c01f in_seq=0 out_seq=0 2024-06-02T15:44:32.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.021+0000 7f6575600700 10 client.25397 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:32.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.021+0000 7f6575600700 1 -- 172.21.15.90:0/302309222 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597af8a3570 con 0x5597af9cd8a0 2024-06-02T15:44:32.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.021+0000 7f6575600700 10 client.25397 handle_command_reply: tid=0 2024-06-02T15:44:32.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.021+0000 7f659d015b80 1 client.25397 shutdown 2024-06-02T15:44:32.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 msgr2=0x5597af95bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65780b5e50 tx=0x7f65780f2ef0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 shutdown_connections 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5597af9cd8a0 0x5597af9cfcc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597af94f7d0 0x5597af96a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597af949880 0x5597af95b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 --2- 172.21.15.90:0/302309222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5597af94eeb0 0x5597af95bd20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 >> 172.21.15.90:0/302309222 conn(0x5597af877be0 msgr2=0x5597af9511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f6575600700 0 client.25397 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:32.022 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 shutdown_connections 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.023+0000 7f659d015b80 1 -- 172.21.15.90:0/302309222 wait complete. 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.024+0000 7f659d015b80 20 client.25397 trim_cache size 0 max 16384 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 msgr2=0x7f65700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 0x7f65700b4dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f658c06ae10 tx=0x7f6584009040 comp rx=0 tx=0).stop 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 msgr2=0x7f658c06deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 0x7f658c06deb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6578005950 tx=0x7f65780074a0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 shutdown_connections 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f65700b2900 0x7f65700b4dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f658c10fff0 0x7f658c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f658c06b250 0x7f658c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 --2- 172.21.15.90:0/2333039993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f658c06bb70 0x7f658c06deb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.025+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 >> 172.21.15.90:0/2333039993 conn(0x7f658c0793f0 msgr2=0x7f658c111480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.027+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 shutdown_connections 2024-06-02T15:44:32.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:32.027+0000 7f6592600700 1 -- 172.21.15.90:0/2333039993 wait complete. 2024-06-02T15:44:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:32 smithi090 ceph-mon[31595]: pgmap v517: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:44:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:32 smithi120 ceph-mon[36451]: pgmap v517: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:44:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:32 smithi135 ceph-mon[36839]: pgmap v517: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-06-02T15:44:33.893 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:44:33.893 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:44:34.334 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:35.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:34 smithi090 ceph-mon[31595]: pgmap v518: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:34 smithi120 ceph-mon[36451]: pgmap v518: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:34 smithi135 ceph-mon[36839]: pgmap v518: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 -- 172.21.15.90:0/4259966616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4109540 msgr2=0x7f2ca4109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 --2- 172.21.15.90:0/4259966616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4109540 0x7f2ca4109920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009990 tx=0x7f2c98009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 -- 172.21.15.90:0/4259966616 shutdown_connections 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 --2- 172.21.15.90:0/4259966616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4103e50 0x7f2ca4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 --2- 172.21.15.90:0/4259966616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4109540 0x7f2ca4109920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 --2- 172.21.15.90:0/4259966616 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 -- 172.21.15.90:0/4259966616 >> 172.21.15.90:0/4259966616 conn(0x7f2ca40fefc0 msgr2=0x7f2ca41013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.680+0000 7f2caa200700 1 -- 172.21.15.90:0/4259966616 shutdown_connections 2024-06-02T15:44:36.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.681+0000 7f2caa200700 1 -- 172.21.15.90:0/4259966616 wait complete. 2024-06-02T15:44:36.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.682+0000 7f2caa200700 1 Processor -- start 2024-06-02T15:44:36.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 -- start start 2024-06-02T15:44:36.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 0x7f2ca410a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4109540 0x7f2ca410ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41136a0 con 0x7f2ca4109540 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4113520 con 0x7f2ca4103490 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.683+0000 7f2caa200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4113820 con 0x7f2ca4103e50 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33028/0 (socket says 172.21.15.90:33028) 2024-06-02T15:44:36.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4187832165 learned_addr learned my addr 172.21.15.90:0/4187832165 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca9800700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4109540 0x7f2ca410ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 0x7f2ca410a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4187832165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 msgr2=0x7f2ca410a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 0x7f2ca410a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4109540 msgr2=0x7f2ca410ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4109540 0x7f2ca410ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.684+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4187832165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca41ac3e0 con 0x7f2ca4103490 2024-06-02T15:44:36.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.685+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c980057c0 tx=0x7f2c980057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.685+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 0x7f2ca410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.685+0000 7f2ca1600700 1 -- 172.21.15.90:0/4187832165 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98027070 con 0x7f2ca4103490 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.685+0000 7f2ca1600700 1 -- 172.21.15.90:0/4187832165 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c98022470 con 0x7f2ca4103490 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.685+0000 7f2ca1600700 1 -- 172.21.15.90:0/4187832165 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017650 con 0x7f2ca4103490 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.686+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca41ac670 con 0x7f2ca4103490 2024-06-02T15:44:36.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.686+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41aca70 con 0x7f2ca4103490 2024-06-02T15:44:36.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.691+0000 7f2ca1600700 1 -- 172.21.15.90:0/4187832165 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2c98009d10 con 0x7f2ca4103490 2024-06-02T15:44:36.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.692+0000 7f2cb6146b80 1 Processor -- start 2024-06-02T15:44:36.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 -- start start 2024-06-02T15:44:36.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 0x56487d527530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d527fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56487d485ec0 con 0x56487d527b50 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56487d475a60 con 0x56487d52d0a0 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.693+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56487d479aa0 con 0x56487d52d9c0 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33042/0 (socket says 172.21.15.90:33042) 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca9800700 1 -- 172.21.15.90:0/2143667476 learned_addr learned my addr 172.21.15.90:0/2143667476 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca1600700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 0x7f2c8c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 0x56487d527530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.694+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d527fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 -- 172.21.15.90:0/2143667476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 msgr2=0x56487d52d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 -- 172.21.15.90:0/2143667476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 msgr2=0x56487d527fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d527fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 -- 172.21.15.90:0/2143667476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487d43e4a0 con 0x56487d52d9c0 2024-06-02T15:44:36.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d527fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:36.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca1600700 1 -- 172.21.15.90:0/4187832165 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c980e2de0 con 0x7f2ca4103490 2024-06-02T15:44:36.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.695+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 0x56487d527530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c980a6100 tx=0x7f2c98077d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4209ab8409be8ca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.696+0000 7f2c96000700 1 -- 172.21.15.90:0/2143667476 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9802b3f0 con 0x56487d52d9c0 2024-06-02T15:44:36.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.696+0000 7f2c96000700 1 -- 172.21.15.90:0/2143667476 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980a9b90 con 0x56487d52d9c0 2024-06-02T15:44:36.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.696+0000 7f2c96000700 1 -- 172.21.15.90:0/2143667476 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980aa780 con 0x56487d52d9c0 2024-06-02T15:44:36.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.696+0000 7f2ca9800700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 0x7f2c8c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.697+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2143667476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 msgr2=0x56487d527530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.697+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 0x56487d527530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2c980a6100 tx=0x7f2c98077d20 comp rx=0 tx=0).stop 2024-06-02T15:44:36.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.697+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2143667476 shutdown_connections 2024-06-02T15:44:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.697+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d527fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.697+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.698+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2143667476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d9c0 0x56487d527530 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.698+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2143667476 >> 172.21.15.90:0/2143667476 conn(0x56487d455c60 msgr2=0x56487d5221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.698+0000 7f2ca9800700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 0x7f2c8c0b4e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009fd0 tx=0x7f2c9c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.698+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2143667476 shutdown_connections 2024-06-02T15:44:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.699+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2143667476 wait complete. 2024-06-02T15:44:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.699+0000 7f2cb6146b80 1 -- start start 2024-06-02T15:44:36.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 Processor -- start 2024-06-02T15:44:36.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d527b50 0x56487d533750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d530af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56487d37c020 con 0x56487d52d9c0 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56487d4167e0 con 0x56487d527b50 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56487d5342a0 con 0x56487d52d0a0 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.700+0000 7f2cb6146b80 10 client.0 _ll_register_callbacks cb 0x56487d41fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.701+0000 7f2ca8e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d527b50 0x56487d533750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.701+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36658/0 (socket says 172.21.15.90:36658) 2024-06-02T15:44:36.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 -- 172.21.15.90:0/2943734117 learned_addr learned my addr 172.21.15.90:0/2943734117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d530af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 -- 172.21.15.90:0/2943734117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 msgr2=0x56487d530af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d530af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 -- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d527b50 msgr2=0x56487d533750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d527b50 0x56487d533750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.702+0000 7f2ca9800700 1 -- 172.21.15.90:0/2943734117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487d3c5950 con 0x56487d52d9c0 2024-06-02T15:44:36.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.703+0000 7f2ca9800700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c017c60 tx=0x7f2c9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03b448525348efe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.703+0000 7f2c8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:36.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.704+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c016c00 con 0x56487d52d9c0 2024-06-02T15:44:36.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.704+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c9c016da0 con 0x56487d52d9c0 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.705+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c980a9b90 con 0x56487d52d9c0 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.705+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c9c02d450 con 0x56487d52d9c0 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.705+0000 7f2cb6146b80 10 client.15264 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.706+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56487d4460e0 con 0x56487d52d9c0 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.706+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980a9b90 con 0x56487d52d9c0 2024-06-02T15:44:36.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c9c022590 con 0x56487d52d9c0 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 10 client.15264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 20 client.15264 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d540dd0 0x56487d543270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 4 client.15264 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:36.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56487d487e10 con 0x56487d540dd0 2024-06-02T15:44:36.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.707+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d540dd0 0x56487d543270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.708+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d540dd0 0x56487d543270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec743da9d52b95d2 server_cookie=c293f9e9f7acd71 in_seq=0 out_seq=0 2024-06-02T15:44:36.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.708+0000 7f2c8aa00700 10 client.15264 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:36.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.709+0000 7f2c8aa00700 1 -- 172.21.15.90:0/2943734117 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56487d487e10 con 0x56487d540dd0 2024-06-02T15:44:36.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.709+0000 7f2c8aa00700 10 client.15264 handle_command_reply: tid=0 2024-06-02T15:44:36.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.709+0000 7f2cb6146b80 1 client.15264 shutdown 2024-06-02T15:44:36.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.709+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 msgr2=0x56487d531030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.709+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c017c60 tx=0x7f2c9c004ad0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 shutdown_connections 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d540dd0 0x56487d543270 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d531030 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d527b50 0x56487d533750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/2943734117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d530af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 >> 172.21.15.90:0/2943734117 conn(0x56487d455c60 msgr2=0x56487d526520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.710+0000 7f2c8aa00700 0 client.15264 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:36.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.711+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 shutdown_connections 2024-06-02T15:44:36.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.711+0000 7f2cb6146b80 1 -- 172.21.15.90:0/2943734117 wait complete. 2024-06-02T15:44:36.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.711+0000 7f2cb6146b80 20 client.15264 trim_cache size 0 max 16384 2024-06-02T15:44:36.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 Processor -- start 2024-06-02T15:44:36.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 -- start start 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 0x56487d527530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d52b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b7fe0 con 0x56487d52d9c0 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b5870 con 0x56487d52d0a0 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.752+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b36c0 con 0x56487d527b50 2024-06-02T15:44:36.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33058/0 (socket says 172.21.15.90:33058) 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca3e00700 1 -- 172.21.15.90:0/4005127388 learned_addr learned my addr 172.21.15.90:0/4005127388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 0x56487d527530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4005127388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 msgr2=0x56487d52e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4005127388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 msgr2=0x56487d52b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d52b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca8e00700 1 -- 172.21.15.90:0/4005127388 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487d53e7a0 con 0x56487d527b50 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.753+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:36.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.754+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 0x56487d527530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009350 tx=0x7f2c98009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f3abb3274c64bf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.754+0000 7f2c95600700 1 -- 172.21.15.90:0/4005127388 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980a97f0 con 0x56487d527b50 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.754+0000 7f2c95600700 1 -- 172.21.15.90:0/4005127388 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980a9b90 con 0x56487d527b50 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.754+0000 7f2c95600700 1 -- 172.21.15.90:0/4005127388 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98022620 con 0x56487d527b50 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 -- 172.21.15.90:0/4005127388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 msgr2=0x56487d527530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 0x56487d527530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009350 tx=0x7f2c98009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 -- 172.21.15.90:0/4005127388 shutdown_connections 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d52d9c0 0x56487d52b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d527b50 0x56487d527530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/4005127388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d0a0 0x56487d52e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 -- 172.21.15.90:0/4005127388 >> 172.21.15.90:0/4005127388 conn(0x56487d455c60 msgr2=0x56487d52bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 -- 172.21.15.90:0/4005127388 shutdown_connections 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.755+0000 7f2cb6146b80 1 -- 172.21.15.90:0/4005127388 wait complete. 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.756+0000 7f2cb6146b80 1 -- start start 2024-06-02T15:44:36.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.756+0000 7f2cb6146b80 1 Processor -- start 2024-06-02T15:44:36.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d539c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d9c0 0x56487d536a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b3d30 con 0x56487d527b50 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b51a0 con 0x56487d52d9c0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca41b4400 con 0x56487d52d0a0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.757+0000 7f2cb6146b80 10 client.0 _ll_register_callbacks cb 0x56487d41fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38430/0 (socket says 172.21.15.90:38430) 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3000140493 learned_addr learned my addr 172.21.15.90:0/3000140493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca9800700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d9c0 0x56487d536a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d9c0 msgr2=0x56487d536a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d9c0 0x56487d536a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3000140493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 msgr2=0x56487d539c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.758+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d539c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d539c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2ca3e00700 1 -- 172.21.15.90:0/3000140493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487d5372c0 con 0x56487d52d0a0 2024-06-02T15:44:36.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d539c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:36.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2ca3e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c90003f50 tx=0x7f2c900098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a785158388d175 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2c8aa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.759+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90004830 con 0x56487d52d0a0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2c900049d0 con 0x56487d52d0a0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c980a9b90 con 0x56487d52d0a0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c90004bd0 con 0x56487d52d0a0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2cb6146b80 10 client.25369 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56487d5acad0 con 0x56487d52d0a0 2024-06-02T15:44:36.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.760+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980a9b90 con 0x56487d52d0a0 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2c90011670 con 0x56487d52d0a0 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 10 client.25369 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 10 client.25369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 20 client.25369 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d5ad1f0 0x56487d5af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 4 client.25369 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56487d4815f0 con 0x56487d5ad1f0 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.761+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d5ad1f0 0x56487d5af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.762+0000 7f2ca8e00700 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d5ad1f0 0x56487d5af610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e811aa1421d396 server_cookie=e98f24dc4340fd4d in_seq=0 out_seq=0 2024-06-02T15:44:36.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.762+0000 7f2c8aa00700 10 client.25369 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:36.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.763+0000 7f2c8aa00700 1 -- 172.21.15.90:0/3000140493 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56487d4815f0 con 0x56487d5ad1f0 2024-06-02T15:44:36.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.763+0000 7f2c8aa00700 10 client.25369 handle_command_reply: tid=0 2024-06-02T15:44:36.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.763+0000 7f2cb6146b80 1 client.25369 shutdown 2024-06-02T15:44:36.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.763+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 msgr2=0x56487d5364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.763+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c90003f50 tx=0x7f2c900098a0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 shutdown_connections 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56487d5ad1f0 0x56487d5af610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56487d52d9c0 0x56487d536a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56487d527b50 0x56487d539c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 --2- 172.21.15.90:0/3000140493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56487d52d0a0 0x56487d5364c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 >> 172.21.15.90:0/3000140493 conn(0x56487d455c60 msgr2=0x56487d52e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.762 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:44:36.762 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:44:36.762 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "e53af33b-18a9-4494-822d-893b68ca70b9", 2024-06-02T15:44:36.762 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:44:36.762 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2c8aa00700 0 client.25369 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 shutdown_connections 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 1 -- 172.21.15.90:0/3000140493 wait complete. 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.764+0000 7f2cb6146b80 20 client.25369 trim_cache size 0 max 16384 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.765+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 msgr2=0x7f2c8c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.765+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 0x7f2c8c0b4e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009fd0 tx=0x7f2c9c009040 comp rx=0 tx=0).stop 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.765+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 msgr2=0x7f2ca410a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.765+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c980057c0 tx=0x7f2c980057f0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.765+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 shutdown_connections 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2c8c0b29d0 0x7f2c8c0b4e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ca4109540 0x7f2ca410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ca4103490 0x7f2ca410a320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 --2- 172.21.15.90:0/4187832165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ca4103e50 0x7f2ca410a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 >> 172.21.15.90:0/4187832165 conn(0x7f2ca40fefc0 msgr2=0x7f2ca4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 shutdown_connections 2024-06-02T15:44:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:36.766+0000 7f2caa200700 1 -- 172.21.15.90:0/4187832165 wait complete. 2024-06-02T15:44:37.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:36 smithi090 ceph-mon[31595]: pgmap v519: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:36 smithi120 ceph-mon[36451]: pgmap v519: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:36 smithi135 ceph-mon[36839]: pgmap v519: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:37.825 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:44:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:37 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:44:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:37 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:37 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:44:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:37 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:37 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:44:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 -- 172.21.15.90:0/3161253016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248106640 msgr2=0x7f6248106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 --2- 172.21.15.90:0/3161253016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248106640 0x7f6248106ac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6240009be0 tx=0x7f624002a6d0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 -- 172.21.15.90:0/3161253016 shutdown_connections 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 --2- 172.21.15.90:0/3161253016 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248106640 0x7f6248106ac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 --2- 172.21.15.90:0/3161253016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248103c90 0x7f6248106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 --2- 172.21.15.90:0/3161253016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248101320 0x7f6248103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 -- 172.21.15.90:0/3161253016 >> 172.21.15.90:0/3161253016 conn(0x7f62480fcf20 msgr2=0x7f62480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 -- 172.21.15.90:0/3161253016 shutdown_connections 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.259+0000 7f624fc00640 1 -- 172.21.15.90:0/3161253016 wait complete. 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.260+0000 7f624fc00640 1 Processor -- start 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 -- start start 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 0x7f624819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248106640 0x7f62481a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6248114c00 con 0x7f6248106640 2024-06-02T15:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6248114a80 con 0x7f6248103c90 2024-06-02T15:44:38.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.261+0000 7f624fc00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6248114d80 con 0x7f6248101320 2024-06-02T15:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.262+0000 7f6247e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.262+0000 7f6247e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33092/0 (socket says 172.21.15.90:33092) 2024-06-02T15:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.262+0000 7f6247e00640 1 -- 172.21.15.90:0/3066358764 learned_addr learned my addr 172.21.15.90:0/3066358764 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.262+0000 7f624d200640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 0x7f624819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.262+0000 7f624dc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248106640 0x7f62481a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 -- 172.21.15.90:0/3066358764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 msgr2=0x7f624819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 0x7f624819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 -- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248106640 msgr2=0x7f62481a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248106640 0x7f62481a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 -- 172.21.15.90:0/3066358764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481a6dd0 con 0x7f6248103c90 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f624d200640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 0x7f624819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6247e00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f623400d950 tx=0x7f623400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6245600640 1 -- 172.21.15.90:0/3066358764 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234017030 con 0x7f6248103c90 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6245600640 1 -- 172.21.15.90:0/3066358764 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6234004020 con 0x7f6248103c90 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f6245600640 1 -- 172.21.15.90:0/3066358764 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234003c30 con 0x7f6248103c90 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.263+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481a70c0 con 0x7f6248103c90 2024-06-02T15:44:38.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.264+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481a75c0 con 0x7f6248103c90 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 Processor -- start 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 -- start start 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76ac3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76ac3690 0x558c76ac5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558c76a2db90 con 0x558c76abdb30 2024-06-02T15:44:38.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c76a1d730 con 0x558c76abe520 2024-06-02T15:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f6252baf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c76a21770 con 0x558c76ac3690 2024-06-02T15:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f624dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f624dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36696/0 (socket says 172.21.15.90:36696) 2024-06-02T15:44:38.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.267+0000 7f624dc00640 1 -- 172.21.15.90:0/2606894019 learned_addr learned my addr 172.21.15.90:0/2606894019 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624d200640 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76ac3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 -- 172.21.15.90:0/2606894019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76ac3690 msgr2=0x558c76ac5b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76ac3690 0x558c76ac5b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 -- 172.21.15.90:0/2606894019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 msgr2=0x558c76ac3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76ac3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 -- 172.21.15.90:0/2606894019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c769b68b0 con 0x558c76abdb30 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.268+0000 7f624dc00640 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f624002acc0 tx=0x7f6240005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500facce29f6b644 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.269+0000 7f623a000640 1 -- 172.21.15.90:0/2606894019 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240041030 con 0x558c76abdb30 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.269+0000 7f623a000640 1 -- 172.21.15.90:0/2606894019 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240030570 con 0x558c76abdb30 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.269+0000 7f623a000640 1 -- 172.21.15.90:0/2606894019 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240036040 con 0x558c76abdb30 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.269+0000 7f6245600640 1 -- 172.21.15.90:0/3066358764 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f623401a070 con 0x7f6248103c90 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 -- 172.21.15.90:0/2606894019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 msgr2=0x558c76abdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f624002acc0 tx=0x7f6240005fd0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 -- 172.21.15.90:0/2606894019 shutdown_connections 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76ac3690 0x558c76ac5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76ac3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 --2- 172.21.15.90:0/2606894019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76abdf50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.270+0000 7f6252baf740 1 -- 172.21.15.90:0/2606894019 >> 172.21.15.90:0/2606894019 conn(0x558c76ab91a0 msgr2=0x558c76ab95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.271+0000 7f6252baf740 1 -- 172.21.15.90:0/2606894019 shutdown_connections 2024-06-02T15:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.271+0000 7f6252baf740 1 -- 172.21.15.90:0/2606894019 wait complete. 2024-06-02T15:44:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.272+0000 7f6252baf740 1 -- start start 2024-06-02T15:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.272+0000 7f6252baf740 1 Processor -- start 2024-06-02T15:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.272+0000 7f6245600640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 0x7f62300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.273+0000 7f6245600640 1 -- 172.21.15.90:0/3066358764 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f62340e1fb0 con 0x7f6248103c90 2024-06-02T15:44:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ac7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76acf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558c76703790 con 0x558c76abe520 2024-06-02T15:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558c769c5bc0 con 0x558c76abdb30 2024-06-02T15:44:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c76ac65e0 con 0x558c76accf80 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6252baf740 10 client.0 _ll_register_callbacks cb 0x558c769a3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f624d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f6247e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ac7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f624d200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33110/0 (socket says 172.21.15.90:33110) 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f624d200640 1 -- 172.21.15.90:0/3572129683 learned_addr learned my addr 172.21.15.90:0/3572129683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.274+0000 7f624dc00640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76acf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 -- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 msgr2=0x558c76ac7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 -- 172.21.15.90:0/3572129683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 msgr2=0x558c76ac7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ac7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 -- 172.21.15.90:0/3572129683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c76732210 con 0x558c76accf80 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f624dc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 0x7f62300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.275+0000 7f6247e00640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ac7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.276+0000 7f624d200640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.276+0000 7f624dc00640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76acf370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6240004af0 tx=0x7f6240003bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cfd9dcf69363a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.276+0000 7f622ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.277+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240041040 con 0x558c76accf80 2024-06-02T15:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.277+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6240004620 con 0x558c76accf80 2024-06-02T15:44:38.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.277+0000 7f624dc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 0x7f62300b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x558c76acf930 tx=0x7f624004d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.278+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6240030570 con 0x558c76accf80 2024-06-02T15:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.278+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6240036040 con 0x558c76accf80 2024-06-02T15:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.278+0000 7f6252baf740 10 client.25378 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.278+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c767fe620 con 0x558c76accf80 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240030570 con 0x558c76accf80 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6240031440 con 0x558c76accf80 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 10 client.25378 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 10 client.25378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 20 client.25378 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76ad3520 0x558c76ad5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 4 client.25378 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.279+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c76a2fae0 con 0x558c76ad3520 2024-06-02T15:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.280+0000 7f624d200640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76ad3520 0x558c76ad5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.281+0000 7f624d200640 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76ad3520 0x558c76ad5910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca5fc97783d4bcd server_cookie=44cac6d8a65f913e in_seq=0 out_seq=0 2024-06-02T15:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.281+0000 7f622ea00640 10 client.25378 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.282+0000 7f622ea00640 1 -- 172.21.15.90:0/3572129683 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558c76a2fae0 con 0x558c76ad3520 2024-06-02T15:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.282+0000 7f622ea00640 10 client.25378 handle_command_reply: tid=0 2024-06-02T15:44:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.282+0000 7f6252baf740 1 client.25378 shutdown 2024-06-02T15:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 msgr2=0x558c76acf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76acf370 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6240004af0 tx=0x7f6240003bc0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 shutdown_connections 2024-06-02T15:44:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76ad3520 0x558c76ad5910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76acf370 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ac7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 --2- 172.21.15.90:0/3572129683 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abdb30 0x558c76ac7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 >> 172.21.15.90:0/3572129683 conn(0x558c76ab91a0 msgr2=0x558c76ab8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.284+0000 7f622ea00640 0 client.25378 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.285+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 shutdown_connections 2024-06-02T15:44:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.285+0000 7f6252baf740 1 -- 172.21.15.90:0/3572129683 wait complete. 2024-06-02T15:44:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.285+0000 7f6252baf740 20 client.25378 trim_cache size 0 max 16384 2024-06-02T15:44:38.299 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.302+0000 7f6252baf740 1 Processor -- start 2024-06-02T15:44:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.302+0000 7f6252baf740 1 -- start start 2024-06-02T15:44:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76ac69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76abc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76abcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b4470 con 0x558c76abdb30 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b1d40 con 0x558c76abe520 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6252baf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62481afb90 con 0x558c76accf80 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f624d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76ac69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6247e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76abc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6247e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76abc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33124/0 (socket says 172.21.15.90:33124) 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f6247e00640 1 -- 172.21.15.90:0/3771882484 learned_addr learned my addr 172.21.15.90:0/3771882484 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.303+0000 7f624dc00640 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76abcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 -- 172.21.15.90:0/3771882484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 msgr2=0x558c76abc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76abc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 -- 172.21.15.90:0/3771882484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 msgr2=0x558c76ac69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76ac69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 -- 172.21.15.90:0/3771882484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c76ab3350 con 0x558c76accf80 2024-06-02T15:44:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624d200640 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76ac69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f624dc00640 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76abcad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f624002abf0 tx=0x7f62400094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800aeb131974fb60 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f6239600640 1 -- 172.21.15.90:0/3771882484 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240041030 con 0x558c76accf80 2024-06-02T15:44:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f6239600640 1 -- 172.21.15.90:0/3771882484 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240030570 con 0x558c76accf80 2024-06-02T15:44:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.305+0000 7f6239600640 1 -- 172.21.15.90:0/3771882484 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624003d460 con 0x558c76accf80 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 -- 172.21.15.90:0/3771882484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 msgr2=0x558c76abcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76abcad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f624002abf0 tx=0x7f62400094a0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 -- 172.21.15.90:0/3771882484 shutdown_connections 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76accf80 0x558c76abcad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76abe520 0x558c76abc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 --2- 172.21.15.90:0/3771882484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abdb30 0x558c76ac69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 -- 172.21.15.90:0/3771882484 >> 172.21.15.90:0/3771882484 conn(0x558c76ab91a0 msgr2=0x558c76abbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.306+0000 7f6252baf740 1 -- 172.21.15.90:0/3771882484 shutdown_connections 2024-06-02T15:44:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.307+0000 7f6252baf740 1 -- 172.21.15.90:0/3771882484 wait complete. 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.307+0000 7f6252baf740 1 -- start start 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.307+0000 7f6252baf740 1 Processor -- start 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76abdb30 0x558c76ae4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 0x558c76ae5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b0200 con 0x558c76abe520 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b1670 con 0x558c76accf80 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b08d0 con 0x558c76abdb30 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6252baf740 10 client.0 _ll_register_callbacks cb 0x7f6240009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6247e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f624dc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 0x558c76ae5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6247e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36734/0 (socket says 172.21.15.90:36734) 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.308+0000 7f6247e00640 1 -- 172.21.15.90:0/677284144 learned_addr learned my addr 172.21.15.90:0/677284144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 -- 172.21.15.90:0/677284144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76abdb30 msgr2=0x558c76ae4830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76abdb30 0x558c76ae4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 -- 172.21.15.90:0/677284144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 msgr2=0x558c76ae4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 -- 172.21.15.90:0/677284144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c76ae6370 con 0x558c76accf80 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f6247e00640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f624dc00640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 0x558c76ae5d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6240031860 tx=0x7f6240003a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6949115bd3028b30 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f622ea00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240041070 con 0x558c76accf80 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6240030570 con 0x558c76accf80 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6240004620 con 0x558c76accf80 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.309+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6240070050 con 0x558c76accf80 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.310+0000 7f6252baf740 10 client.25430 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.310+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c76ae6370 con 0x558c76accf80 2024-06-02T15:44:38.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:37 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:44:38.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:37 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:37 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:44:38.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:37 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.310 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:37 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.310+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240030570 con 0x558c76accf80 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f624003d9a0 con 0x558c76accf80 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 10 client.25430 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 20 client.25430 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76b37e00 0x558c76b3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 4 client.25430 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c76a292c0 con 0x558c76b37e00 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.311+0000 7f624d200640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76b37e00 0x558c76b3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.312+0000 7f622ea00640 10 client.25430 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.312+0000 7f624d200640 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76b37e00 0x558c76b3a220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4097e58269cc47 server_cookie=97a9a923289a75e0 in_seq=0 out_seq=0 2024-06-02T15:44:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.317+0000 7f622ea00640 1 -- 172.21.15.90:0/677284144 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c76a292c0 con 0x558c76b37e00 2024-06-02T15:44:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.317+0000 7f622ea00640 10 client.25430 handle_command_reply: tid=0 2024-06-02T15:44:38.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.317+0000 7f6252baf740 1 client.25430 shutdown 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 msgr2=0x558c76ae5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 0x558c76ae5d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6240031860 tx=0x7f6240003a60 comp rx=0 tx=0).stop 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 shutdown_connections 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558c76b37e00 0x558c76b3a220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558c76accf80 0x558c76ae5d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c76abe520 0x558c76ae4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 --2- 172.21.15.90:0/677284144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c76abdb30 0x558c76ae4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 >> 172.21.15.90:0/677284144 conn(0x558c76ab91a0 msgr2=0x558c76ab8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f622ea00640 0 client.25430 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 shutdown_connections 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.318+0000 7f6252baf740 1 -- 172.21.15.90:0/677284144 wait complete. 2024-06-02T15:44:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.319+0000 7f6252baf740 20 client.25430 trim_cache size 0 max 16384 2024-06-02T15:44:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:37 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:44:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:37 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:37 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:44:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:37 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:44:38.317 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:37 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:44:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:44:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.319+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 msgr2=0x7f62300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.319+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 0x7f62300b4dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x558c76acf930 tx=0x7f624004d000 comp rx=0 tx=0).stop 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.319+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 msgr2=0x7f624819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.319+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f623400d950 tx=0x7f623400de10 comp rx=0 tx=0).stop 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 shutdown_connections 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62300b2900 0x7f62300b4dc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6248106640 0x7f62481a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6248103c90 0x7f624819f6f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 --2- 172.21.15.90:0/3066358764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6248101320 0x7f624819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.320+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 >> 172.21.15.90:0/3066358764 conn(0x7f62480fcf20 msgr2=0x7f6248101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.321+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 shutdown_connections 2024-06-02T15:44:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:38.321+0000 7f624fc00640 1 -- 172.21.15.90:0/3066358764 wait complete. 2024-06-02T15:44:39.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:38 smithi090 ceph-mon[31595]: pgmap v520: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:38 smithi120 ceph-mon[36451]: pgmap v520: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:38 smithi135 ceph-mon[36839]: pgmap v520: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 -- 172.21.15.90:0/288015745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 msgr2=0x7efd281081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 --2- 172.21.15.90:0/288015745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd281081c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd14009990 tx=0x7efd14009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 -- 172.21.15.90:0/288015745 shutdown_connections 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 --2- 172.21.15.90:0/288015745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281088f0 0x7efd28110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 --2- 172.21.15.90:0/288015745 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281037f0 0x7efd28103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 --2- 172.21.15.90:0/288015745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd281081c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.647+0000 7efd2e600700 1 -- 172.21.15.90:0/288015745 >> 172.21.15.90:0/288015745 conn(0x7efd2807a6a0 msgr2=0x7efd2807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.649+0000 7efd2e600700 1 -- 172.21.15.90:0/288015745 shutdown_connections 2024-06-02T15:44:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.649+0000 7efd2e600700 1 -- 172.21.15.90:0/288015745 wait complete. 2024-06-02T15:44:40.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.650+0000 7efd2e600700 1 Processor -- start 2024-06-02T15:44:40.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.650+0000 7efd2e600700 1 -- start start 2024-06-02T15:44:40.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd2819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281088f0 0x7efd281a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd28113720 con 0x7efd281037f0 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd281135a0 con 0x7efd281088f0 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd2e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd281138a0 con 0x7efd281041a0 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd27400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd27400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36746/0 (socket says 172.21.15.90:36746) 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.651+0000 7efd27400700 1 -- 172.21.15.90:0/2190306451 learned_addr learned my addr 172.21.15.90:0/2190306451 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27e00700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281088f0 0x7efd281a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd26a00700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd2819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 -- 172.21.15.90:0/2190306451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 msgr2=0x7efd2819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd2819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 -- 172.21.15.90:0/2190306451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281088f0 msgr2=0x7efd281a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281088f0 0x7efd281a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 -- 172.21.15.90:0/2190306451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281a0e00 con 0x7efd281037f0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.652+0000 7efd27400700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e9c0 tx=0x7efd1800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.653+0000 7efd1fe00700 1 -- 172.21.15.90:0/2190306451 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800cbd0 con 0x7efd281037f0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.653+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd281a1090 con 0x7efd281037f0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.653+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2810a320 con 0x7efd281037f0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.653+0000 7efd1fe00700 1 -- 172.21.15.90:0/2190306451 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd1800cd70 con 0x7efd281037f0 2024-06-02T15:44:40.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.653+0000 7efd1fe00700 1 -- 172.21.15.90:0/2190306451 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18005610 con 0x7efd281037f0 2024-06-02T15:44:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 Processor -- start 2024-06-02T15:44:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 -- start start 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95998290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd1fe00700 1 -- 172.21.15.90:0/2190306451 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd18004750 con 0x7efd281037f0 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f958f0e40 con 0x561f95997eb0 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f958e09e0 con 0x561f95992880 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.659+0000 7efd38f54b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f958e4a20 con 0x561f959987d0 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95998290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38512/0 (socket says 172.21.15.90:38512) 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95998290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36750/0 (socket says 172.21.15.90:36750) 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 -- 172.21.15.90:0/2347042770 learned_addr learned my addr 172.21.15.90:0/2347042770 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 -- 172.21.15.90:0/2347042770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 msgr2=0x561f95992ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 -- 172.21.15.90:0/2347042770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 msgr2=0x561f95998290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95998290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.660+0000 7efd27400700 1 -- 172.21.15.90:0/2347042770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f958a9420 con 0x561f959987d0 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.661+0000 7efd27400700 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd180e8a80 tx=0x7efd180e8ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cef949d164dfdc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.661+0000 7efd1cc00700 1 -- 172.21.15.90:0/2347042770 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180f1070 con 0x561f959987d0 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.662+0000 7efd1cc00700 1 -- 172.21.15.90:0/2347042770 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd180eae00 con 0x561f959987d0 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.662+0000 7efd1cc00700 1 -- 172.21.15.90:0/2347042770 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180e4070 con 0x561f959987d0 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.662+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 0x7efd0c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.662+0000 7efd1fe00700 1 -- 172.21.15.90:0/2190306451 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efd180e2760 con 0x7efd281037f0 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 -- 172.21.15.90:0/2347042770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 msgr2=0x561f95992260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd180e8a80 tx=0x7efd180e8ab0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd27e00700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 0x7efd0c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 -- 172.21.15.90:0/2347042770 shutdown_connections 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95998290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2347042770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f95992260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 -- 172.21.15.90:0/2347042770 >> 172.21.15.90:0/2347042770 conn(0x561f958c0be0 msgr2=0x561f9598d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.663+0000 7efd38f54b80 1 -- 172.21.15.90:0/2347042770 shutdown_connections 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.664+0000 7efd38f54b80 1 -- 172.21.15.90:0/2347042770 wait complete. 2024-06-02T15:44:40.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.664+0000 7efd27e00700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 0x7efd0c0b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd20009440 tx=0x7efd2000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.664+0000 7efd38f54b80 1 -- start start 2024-06-02T15:44:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.665+0000 7efd38f54b80 1 Processor -- start 2024-06-02T15:44:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.665+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.665+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f9599b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f9599b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f9599b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36766/0 (socket says 172.21.15.90:36766) 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38524/0 (socket says 172.21.15.90:38524) 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f9599bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd26a00700 1 -- 172.21.15.90:0/1879735483 learned_addr learned my addr 172.21.15.90:0/1879735483 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f957e6fa0 con 0x561f95997eb0 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f95881760 con 0x561f959987d0 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f9599f240 con 0x561f95992880 2024-06-02T15:44:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd38f54b80 10 client.0 _ll_register_callbacks cb 0x561f9588ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.666+0000 7efd27e00700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f9599bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 -- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 msgr2=0x561f9599bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f9599bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 -- 172.21.15.90:0/1879735483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 msgr2=0x561f9599b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f9599b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f958308d0 con 0x561f95992880 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd27400700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd18009fd0 tx=0x7efd180eafb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5c662b8efe0f24e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd11600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180f1070 con 0x561f95992880 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efd180e2470 con 0x561f95992880 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.667+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd180eae00 con 0x561f95992880 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.668+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd180a5890 con 0x561f95992880 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.668+0000 7efd38f54b80 10 client.25393 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.668+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f958b1060 con 0x561f95992880 2024-06-02T15:44:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.668+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd180eae00 con 0x561f95992880 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.669+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efd180a9c30 con 0x561f95992880 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.669+0000 7efd38f54b80 10 client.25393 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.669+0000 7efd38f54b80 10 client.25393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.669+0000 7efd38f54b80 20 client.25393 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.669+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f959abc10 0x561f959ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.670+0000 7efd38f54b80 4 client.25393 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:40.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.670+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f958f2d90 con 0x561f959abc10 2024-06-02T15:44:40.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.670+0000 7efd26a00700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f959abc10 0x561f959ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.671+0000 7efd11600700 10 client.25393 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:40.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.671+0000 7efd26a00700 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f959abc10 0x561f959ae000 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de4ffb4d30df6d3 server_cookie=89f076456c0a19a0 in_seq=0 out_seq=0 2024-06-02T15:44:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.671+0000 7efd11600700 1 -- 172.21.15.90:0/1879735483 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f958f2d90 con 0x561f959abc10 2024-06-02T15:44:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.671+0000 7efd11600700 10 client.25393 handle_command_reply: tid=0 2024-06-02T15:44:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 client.25393 shutdown 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 msgr2=0x561f9599e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd18009fd0 tx=0x7efd180eafb0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 shutdown_connections 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f959abc10 0x561f959ae000 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f9599bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f9599e4a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 --2- 172.21.15.90:0/1879735483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f9599b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd11600700 0 client.25393 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.672+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 >> 172.21.15.90:0/1879735483 conn(0x561f958c0be0 msgr2=0x561f95991250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.673+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 shutdown_connections 2024-06-02T15:44:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.673+0000 7efd38f54b80 1 -- 172.21.15.90:0/1879735483 wait complete. 2024-06-02T15:44:40.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.673+0000 7efd38f54b80 20 client.25393 trim_cache size 0 max 16384 2024-06-02T15:44:40.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.695+0000 7efd38f54b80 1 Processor -- start 2024-06-02T15:44:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.695+0000 7efd38f54b80 1 -- start start 2024-06-02T15:44:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f9598c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd08011400 con 0x561f95997eb0 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd0800ec90 con 0x561f95992880 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd38f54b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd0800cae0 con 0x561f959987d0 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f9598c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd27400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd27e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f9598c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38532/0 (socket says 172.21.15.90:38532) 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd27e00700 1 -- 172.21.15.90:0/2162162646 learned_addr learned my addr 172.21.15.90:0/2162162646 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.696+0000 7efd26a00700 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36770/0 (socket says 172.21.15.90:36770) 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 -- 172.21.15.90:0/2162162646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 msgr2=0x561f9598c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f9598c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 -- 172.21.15.90:0/2162162646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 msgr2=0x561f95986680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 -- 172.21.15.90:0/2162162646 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9598dd30 con 0x561f95992880 2024-06-02T15:44:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd26a00700 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd27400700 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd180e89d0 tx=0x7efd180a9ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b39cb3bb1ddb7e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd13e00700 1 -- 172.21.15.90:0/2162162646 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180f1030 con 0x561f95992880 2024-06-02T15:44:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd13e00700 1 -- 172.21.15.90:0/2162162646 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd180cc2f0 con 0x561f95992880 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.697+0000 7efd13e00700 1 -- 172.21.15.90:0/2162162646 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd180a5160 con 0x561f95992880 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 -- 172.21.15.90:0/2162162646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 msgr2=0x561f95992260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efd180e89d0 tx=0x7efd180a9ff0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 -- 172.21.15.90:0/2162162646 shutdown_connections 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f959987d0 0x561f9598c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f95992880 0x561f95992260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2162162646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f95986680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.698+0000 7efd38f54b80 1 -- 172.21.15.90:0/2162162646 >> 172.21.15.90:0/2162162646 conn(0x561f958c0be0 msgr2=0x561f9599a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.699+0000 7efd38f54b80 1 -- 172.21.15.90:0/2162162646 shutdown_connections 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.699+0000 7efd38f54b80 1 -- 172.21.15.90:0/2162162646 wait complete. 2024-06-02T15:44:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.700+0000 7efd38f54b80 1 -- start start 2024-06-02T15:44:40.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.700+0000 7efd38f54b80 1 Processor -- start 2024-06-02T15:44:40.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.700+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f959b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.700+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f959b3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd38f54b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f959b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd38f54b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd0800d150 con 0x561f95997eb0 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd38f54b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd0800e5c0 con 0x561f959987d0 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd38f54b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd0800d820 con 0x561f95992880 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd38f54b80 10 client.0 _ll_register_callbacks cb 0x561f9588ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f959b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd27400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f959b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38538/0 (socket says 172.21.15.90:38538) 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd27400700 1 -- 172.21.15.90:0/2604559117 learned_addr learned my addr 172.21.15.90:0/2604559117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.701+0000 7efd26a00700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f959b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.702+0000 7efd26a00700 1 -- 172.21.15.90:0/2604559117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 msgr2=0x561f959b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.702+0000 7efd26a00700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f959b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.702+0000 7efd26a00700 1 -- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 msgr2=0x561f959b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.702+0000 7efd26a00700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f959b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.702+0000 7efd26a00700 1 -- 172.21.15.90:0/2604559117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f95a15630 con 0x561f95997eb0 2024-06-02T15:44:40.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.703+0000 7efd26a00700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f959b3780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd1400b2e0 tx=0x7efd14005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9071aa01ee9ce8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:40.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.703+0000 7efd11600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:40.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.703+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1400b7f0 con 0x561f95997eb0 2024-06-02T15:44:40.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.703+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efd1400b990 con 0x561f95997eb0 2024-06-02T15:44:40.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.703+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd180cc2f0 con 0x561f95997eb0 2024-06-02T15:44:40.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.704+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd1400bb90 con 0x561f95997eb0 2024-06-02T15:44:40.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.704+0000 7efd38f54b80 10 client.15300 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:40.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.704+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f95a16900 con 0x561f95997eb0 2024-06-02T15:44:40.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd180cc2f0 con 0x561f95997eb0 2024-06-02T15:44:40.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7efd1400f460 con 0x561f95997eb0 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 20 client.15300 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f95a16fe0 0x561f95a19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 4 client.15300 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.705+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f958ec570 con 0x561f95a16fe0 2024-06-02T15:44:40.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.706+0000 7efd27400700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f95a16fe0 0x561f95a19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.706+0000 7efd27400700 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f95a16fe0 0x561f95a19400 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83693eead7a0819f server_cookie=52b698a3d8e566d9 in_seq=0 out_seq=0 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.706+0000 7efd11600700 10 client.15300 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd11600700 1 -- 172.21.15.90:0/2604559117 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f958ec570 con 0x561f95a16fe0 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd11600700 10 client.15300 handle_command_reply: tid=0 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 client.15300 shutdown 2024-06-02T15:44:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 msgr2=0x561f959b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f959b3780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd1400b2e0 tx=0x7efd14005050 comp rx=0 tx=0).stop 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 shutdown_connections 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f95a16fe0 0x561f95a19400 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f959987d0 0x561f959b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f95992880 0x561f959b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 --2- 172.21.15.90:0/2604559117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f95997eb0 0x561f959b3780 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 >> 172.21.15.90:0/2604559117 conn(0x561f958c0be0 msgr2=0x561f95998bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.707+0000 7efd11600700 0 client.15300 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.708+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 shutdown_connections 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.708+0000 7efd38f54b80 1 -- 172.21.15.90:0/2604559117 wait complete. 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.708+0000 7efd38f54b80 20 client.15300 trim_cache size 0 max 16384 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:44:40.706 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 msgr2=0x7efd0c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 0x7efd0c0b4dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efd20009440 tx=0x7efd2000b040 comp rx=0 tx=0).stop 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 msgr2=0x7efd2819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e9c0 tx=0x7efd1800ecd0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 shutdown_connections 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efd0c0b2900 0x7efd0c0b4dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd281088f0 0x7efd281a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd281037f0 0x7efd2819bdf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 --2- 172.21.15.90:0/2190306451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd281041a0 0x7efd2819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.709+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 >> 172.21.15.90:0/2190306451 conn(0x7efd2807a6a0 msgr2=0x7efd28101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.710+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 shutdown_connections 2024-06-02T15:44:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:40.710+0000 7efd2e600700 1 -- 172.21.15.90:0/2190306451 wait complete. 2024-06-02T15:44:41.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:40 smithi090 ceph-mon[31595]: pgmap v521: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:44:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:40 smithi120 ceph-mon[36451]: pgmap v521: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:44:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:40 smithi135 ceph-mon[36839]: pgmap v521: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-06-02T15:44:41.600 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:44:41.600 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:44:42.077 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:43 smithi120 ceph-mon[36451]: pgmap v522: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:44:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:43 smithi135 ceph-mon[36839]: pgmap v522: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:44:43.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:43 smithi090 ceph-mon[31595]: pgmap v522: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-06-02T15:44:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.514+0000 7f8cf7c00700 1 -- 172.21.15.90:0/2429025675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0108500 msgr2=0x7f8cf01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:44.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.514+0000 7f8cf7c00700 1 --2- 172.21.15.90:0/2429025675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0108500 0x7f8cf01088e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009990 tx=0x7f8ce4009ca0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 -- 172.21.15.90:0/2429025675 shutdown_connections 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 --2- 172.21.15.90:0/2429025675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf0103f90 0x7f8cf0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 --2- 172.21.15.90:0/2429025675 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0108500 0x7f8cf01088e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 --2- 172.21.15.90:0/2429025675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf01035f0 0x7f8cf0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 -- 172.21.15.90:0/2429025675 >> 172.21.15.90:0/2429025675 conn(0x7f8cf007a6a0 msgr2=0x7f8cf007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.515+0000 7f8cf7c00700 1 -- 172.21.15.90:0/2429025675 shutdown_connections 2024-06-02T15:44:44.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.516+0000 7f8cf7c00700 1 -- 172.21.15.90:0/2429025675 wait complete. 2024-06-02T15:44:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 Processor -- start 2024-06-02T15:44:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 -- start start 2024-06-02T15:44:44.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:44.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 0x7f8cf006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:44.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 0x7f8cf006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:44.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf01137a0 con 0x7f8cf01035f0 2024-06-02T15:44:44.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.517+0000 7f8cf7c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0113620 con 0x7f8cf0103f90 2024-06-02T15:44:44.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf7c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0113920 con 0x7f8cf0108500 2024-06-02T15:44:44.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51226/0 (socket says 172.21.15.90:51226) 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5200700 1 -- 172.21.15.90:0/3149766936 learned_addr learned my addr 172.21.15.90:0/3149766936 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5c00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 0x7f8cf006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5c00700 1 -- 172.21.15.90:0/3149766936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 msgr2=0x7f8cf006de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cefe00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 0x7f8cf006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5c00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 0x7f8cf006de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5c00700 1 -- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 msgr2=0x7f8cf0074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.518+0000 7f8cf5c00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cf5c00700 1 -- 172.21.15.90:0/3149766936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf006e9c0 con 0x7f8cf0108500 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cf5200700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cf5c00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 0x7f8cf006e380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007f40 tx=0x7f8ce800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:44.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cefe00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 0x7f8cf006de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8004db0 con 0x7f8cf0108500 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cf7c00700 1 -- 172.21.15.90:0/3149766936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf0072890 con 0x7f8cf0108500 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce80093a0 con 0x7f8cf0108500 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8cf7c00700 1 -- 172.21.15.90:0/3149766936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf0072dd0 con 0x7f8cf0108500 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.519+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8019020 con 0x7f8cf0108500 2024-06-02T15:44:44.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.521+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd0005330 con 0x7f8cf0108500 2024-06-02T15:44:44.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.523+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8ce801c020 con 0x7f8cf0108500 2024-06-02T15:44:44.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.524+0000 7f8ced600700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 0x7f8cd80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:44.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.525+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8ce80e1c20 con 0x7f8cf0108500 2024-06-02T15:44:44.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.525+0000 7f8cf5200700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 0x7f8cd80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:44.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.526+0000 7f8cf5200700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 0x7f8cd80b4e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009750 tx=0x7f8ce401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:44.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.527+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ce8010070 con 0x7f8cf0108500 2024-06-02T15:44:44.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.855+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cd0005620 con 0x7f8cf0108500 2024-06-02T15:44:44.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.856+0000 7f8ced600700 1 -- 172.21.15.90:0/3149766936 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11877 (secure 0 0 0) 0x7f8ce80a9530 con 0x7f8cf0108500 2024-06-02T15:44:44.855 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:44:44.855 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:43:15.667079+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:44:44.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 msgr2=0x7f8cd80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 0x7f8cd80b4e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009750 tx=0x7f8ce401a040 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 msgr2=0x7f8cf006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 0x7f8cf006e380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007f40 tx=0x7f8ce800d3b0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 shutdown_connections 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8cd80b2980 0x7f8cd80b4e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cf0108500 0x7f8cf006e380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf01035f0 0x7f8cf0074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 --2- 172.21.15.90:0/3149766936 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cf0103f90 0x7f8cf006de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:44.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.859+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 >> 172.21.15.90:0/3149766936 conn(0x7f8cf007a6a0 msgr2=0x7f8cf01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:44.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.860+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 shutdown_connections 2024-06-02T15:44:44.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:44.860+0000 7f8ce2a00700 1 -- 172.21.15.90:0/3149766936 wait complete. 2024-06-02T15:44:44.859 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-06-02T15:44:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:45 smithi120 ceph-mon[36451]: pgmap v523: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:45 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3149766936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:45 smithi135 ceph-mon[36839]: pgmap v523: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:45 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3149766936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:45 smithi090 ceph-mon[31595]: pgmap v523: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:45 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3149766936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:44:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:47 smithi120 ceph-mon[36451]: pgmap v524: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:47 smithi135 ceph-mon[36839]: pgmap v524: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:47.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:47 smithi090 ceph-mon[31595]: pgmap v524: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:47.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:44:48.397 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:49 smithi120 ceph-mon[36451]: pgmap v525: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:49 smithi135 ceph-mon[36839]: pgmap v525: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:49.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:49 smithi090 ceph-mon[31595]: pgmap v525: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:51.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 -- 172.21.15.90:0/2440932310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 msgr2=0x7fe2d0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 --2- 172.21.15.90:0/2440932310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0110470 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009fc0 tx=0x7fe2c800b330 comp rx=0 tx=0).stop 2024-06-02T15:44:51.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 -- 172.21.15.90:0/2440932310 shutdown_connections 2024-06-02T15:44:51.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 --2- 172.21.15.90:0/2440932310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0110470 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 --2- 172.21.15.90:0/2440932310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 --2- 172.21.15.90:0/2440932310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 0x7fe2d01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 -- 172.21.15.90:0/2440932310 >> 172.21.15.90:0/2440932310 conn(0x7fe2d007a6a0 msgr2=0x7fe2d007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.118+0000 7fe2d7400700 1 -- 172.21.15.90:0/2440932310 shutdown_connections 2024-06-02T15:44:51.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.119+0000 7fe2d7400700 1 -- 172.21.15.90:0/2440932310 wait complete. 2024-06-02T15:44:51.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.120+0000 7fe2d7400700 1 Processor -- start 2024-06-02T15:44:51.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.120+0000 7fe2d7400700 1 -- start start 2024-06-02T15:44:51.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.120+0000 7fe2d7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 0x7fe2d019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d7400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d7400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0113710 con 0x7fe2d01037f0 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d7400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0113590 con 0x7fe2d01088f0 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d7400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0113890 con 0x7fe2d01041a0 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58678/0 (socket says 172.21.15.90:58678) 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d4a00700 1 -- 172.21.15.90:0/43346518 learned_addr learned my addr 172.21.15.90:0/43346518 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 0x7fe2d019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.121+0000 7fe2d5400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 -- 172.21.15.90:0/43346518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 msgr2=0x7fe2d019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 0x7fe2d019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 -- 172.21.15.90:0/43346518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 msgr2=0x7fe2d0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 -- 172.21.15.90:0/43346518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d019a0f0 con 0x7fe2d01037f0 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400ec10 tx=0x7fe2c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2cd600700 1 -- 172.21.15.90:0/43346518 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400cc90 con 0x7fe2d01037f0 2024-06-02T15:44:51.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d019a3e0 con 0x7fe2d01037f0 2024-06-02T15:44:51.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2cd600700 1 -- 172.21.15.90:0/43346518 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c400ce30 con 0x7fe2d01037f0 2024-06-02T15:44:51.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2cd600700 1 -- 172.21.15.90:0/43346518 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4010430 con 0x7fe2d01037f0 2024-06-02T15:44:51.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.122+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a66a0 con 0x7fe2d01037f0 2024-06-02T15:44:51.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.126+0000 7fe2cd600700 1 -- 172.21.15.90:0/43346518 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe2c4004000 con 0x7fe2d01037f0 2024-06-02T15:44:51.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.127+0000 7fe2e1d50b80 1 Processor -- start 2024-06-02T15:44:51.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 -- start start 2024-06-02T15:44:51.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc22dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cdc2235e40 con 0x55cdc22db850 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cdc22259e0 con 0x55cdc22e0430 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cdc2229a20 con 0x55cdc22dc170 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2d5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.128+0000 7fe2d5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58682/0 (socket says 172.21.15.90:58682) 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d5400700 1 -- 172.21.15.90:0/1100369994 learned_addr learned my addr 172.21.15.90:0/1100369994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc22dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2cd600700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 0x7fe2b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d5400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 0x7fe2b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 -- 172.21.15.90:0/1100369994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 msgr2=0x55cdc22e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 -- 172.21.15.90:0/1100369994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 msgr2=0x55cdc22dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d4a00700 1 -- 172.21.15.90:0/1100369994 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc21ee420 con 0x55cdc22dc170 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2d5400700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.129+0000 7fe2cd600700 1 -- 172.21.15.90:0/43346518 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe2c40e2810 con 0x7fe2d01037f0 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.130+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc22dfef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a6c90 tx=0x7fe2c4021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a549b164fd16acee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.130+0000 7fe2c2000700 1 -- 172.21.15.90:0/1100369994 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a9190 con 0x55cdc22dc170 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.130+0000 7fe2c2000700 1 -- 172.21.15.90:0/1100369994 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c40a9530 con 0x55cdc22dc170 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.130+0000 7fe2c2000700 1 -- 172.21.15.90:0/1100369994 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a9a20 con 0x55cdc22dc170 2024-06-02T15:44:51.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.130+0000 7fe2d5400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 0x7fe2b80b4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009790 tx=0x7fe2c800b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/1100369994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 msgr2=0x55cdc22dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc22dfef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a6c90 tx=0x7fe2c4021000 comp rx=0 tx=0).stop 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/1100369994 shutdown_connections 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/1100369994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc22dfef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/1100369994 >> 172.21.15.90:0/1100369994 conn(0x55cdc2205be0 msgr2=0x55cdc22d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/1100369994 shutdown_connections 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.131+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/1100369994 wait complete. 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.132+0000 7fe2e1d50b80 1 -- start start 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.132+0000 7fe2e1d50b80 1 Processor -- start 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22e0430 0x55cdc22e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cdc212bfa0 con 0x55cdc22e0430 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55cdc21c6760 con 0x55cdc22dc170 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cdc22e85a0 con 0x55cdc22db850 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2e1d50b80 10 client.0 _ll_register_callbacks cb 0x55cdc21cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38284/0 (socket says 172.21.15.90:38284) 2024-06-02T15:44:51.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 -- 172.21.15.90:0/679646040 learned_addr learned my addr 172.21.15.90:0/679646040 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d5400700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22e0430 0x55cdc22e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 -- 172.21.15.90:0/679646040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 msgr2=0x55cdc22e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.133+0000 7fe2d4a00700 1 -- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22e0430 msgr2=0x55cdc22e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22e0430 0x55cdc22e51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2d4a00700 1 -- 172.21.15.90:0/679646040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc21758d0 con 0x55cdc22db850 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a99b0 tx=0x7fe2c4003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b4cfccbe99f2d47 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a9190 con 0x55cdc22db850 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe2c40a9c80 con 0x55cdc22db850 2024-06-02T15:44:51.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c40a9530 con 0x55cdc22db850 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.134+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2c40aa120 con 0x55cdc22db850 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 10 client.25417 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cdc21f6060 con 0x55cdc22db850 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c40a9530 con 0x55cdc22db850 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe2c40b23f0 con 0x55cdc22db850 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 10 client.25417 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 10 client.25417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 20 client.25417 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc22f4f10 0x55cdc22f7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 4 client.25417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.135+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc2237d90 con 0x55cdc22f4f10 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.136+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc22f4f10 0x55cdc22f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.136+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc22f4f10 0x55cdc22f7300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e62c4e4534e385f server_cookie=cfac42edae0e46a9 in_seq=0 out_seq=0 2024-06-02T15:44:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.136+0000 7fe2b6a00700 10 client.25417 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:51.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.137+0000 7fe2b6a00700 1 -- 172.21.15.90:0/679646040 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cdc2237d90 con 0x55cdc22f4f10 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.137+0000 7fe2b6a00700 10 client.25417 handle_command_reply: tid=0 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.137+0000 7fe2e1d50b80 1 client.25417 shutdown 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 msgr2=0x55cdc22e78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a99b0 tx=0x7fe2c4003a50 comp rx=0 tx=0).stop 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 shutdown_connections 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc22f4f10 0x55cdc22f7300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22e0430 0x55cdc22e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22db850 0x55cdc22e78f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/679646040 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2b6a00700 0 client.25417 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 >> 172.21.15.90:0/679646040 conn(0x55cdc2205be0 msgr2=0x55cdc22d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 shutdown_connections 2024-06-02T15:44:51.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/679646040 wait complete. 2024-06-02T15:44:51.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.138+0000 7fe2e1d50b80 20 client.25417 trim_cache size 0 max 16384 2024-06-02T15:44:51.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.171+0000 7fe2e1d50b80 1 Processor -- start 2024-06-02T15:44:51.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.171+0000 7fe2e1d50b80 1 -- start start 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.171+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22e0430 0x55cdc22df250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01afac0 con 0x55cdc22db850 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01ad350 con 0x55cdc22dc170 2024-06-02T15:44:51.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01ab1a0 con 0x55cdc22e0430 2024-06-02T15:44:51.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58710/0 (socket says 172.21.15.90:58710) 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2d4a00700 1 -- 172.21.15.90:0/498926039 learned_addr learned my addr 172.21.15.90:0/498926039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.172+0000 7fe2d5400700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22e0430 0x55cdc22df250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 -- 172.21.15.90:0/498926039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22e0430 msgr2=0x55cdc22df250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22e0430 0x55cdc22df250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 -- 172.21.15.90:0/498926039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 msgr2=0x55cdc22e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 -- 172.21.15.90:0/498926039 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc22e8230 con 0x55cdc22dc170 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:51.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e6a50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55cdc21c6760 tx=0x7fe2bc004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88da1086a57ff24c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2c1600700 1 -- 172.21.15.90:0/498926039 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc003a00 con 0x55cdc22dc170 2024-06-02T15:44:51.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2c1600700 1 -- 172.21.15.90:0/498926039 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc003ba0 con 0x55cdc22dc170 2024-06-02T15:44:51.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.173+0000 7fe2c1600700 1 -- 172.21.15.90:0/498926039 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0126b0 con 0x55cdc22dc170 2024-06-02T15:44:51.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/498926039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 msgr2=0x55cdc22e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e6a50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55cdc21c6760 tx=0x7fe2bc004320 comp rx=0 tx=0).stop 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/498926039 shutdown_connections 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22e0430 0x55cdc22df250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc22e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/498926039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22dc170 0x55cdc22e6a50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/498926039 >> 172.21.15.90:0/498926039 conn(0x55cdc2205be0 msgr2=0x55cdc22e6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/498926039 shutdown_connections 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.174+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/498926039 wait complete. 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.175+0000 7fe2e1d50b80 1 -- start start 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.175+0000 7fe2e1d50b80 1 Processor -- start 2024-06-02T15:44:51.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc23585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01ab810 con 0x55cdc22db850 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01acc80 con 0x55cdc22e0430 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01abee0 con 0x55cdc22dc170 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2e1d50b80 10 client.0 _ll_register_callbacks cb 0x55cdc21cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2d4a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58720/0 (socket says 172.21.15.90:58720) 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2d4a00700 1 -- 172.21.15.90:0/4163704663 learned_addr learned my addr 172.21.15.90:0/4163704663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2d5400700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc23585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.176+0000 7fe2d4a00700 1 -- 172.21.15.90:0/4163704663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 msgr2=0x55cdc23585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc23585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2d4a00700 1 -- 172.21.15.90:0/4163704663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 msgr2=0x55cdc22f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2d4a00700 1 -- 172.21.15.90:0/4163704663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc22fa650 con 0x55cdc22db850 2024-06-02T15:44:51.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2d4a00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a99b0 tx=0x7fe2c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55362086d44bda07 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:51.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a9190 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe2c40a9e80 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2bc003ba0 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2c40a9530 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.177+0000 7fe2e1d50b80 10 client.15324 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cdc235f8e0 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc003ba0 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe2c40079b0 con 0x55cdc22db850 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 10 client.15324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 20 client.15324 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc235fdd0 0x55cdc23621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 4 client.15324 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:44:51.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.178+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdc2231570 con 0x55cdc235fdd0 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc235fdd0 0x55cdc23621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2b6a00700 10 client.15324 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2cfe00700 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc235fdd0 0x55cdc23621f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=374aa6d82ef5c019 server_cookie=2e44607b5d23c8f5 in_seq=0 out_seq=0 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2b6a00700 1 -- 172.21.15.90:0/4163704663 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdc2231570 con 0x55cdc235fdd0 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2b6a00700 10 client.15324 handle_command_reply: tid=0 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.179+0000 7fe2e1d50b80 1 client.15324 shutdown 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 msgr2=0x55cdc2358090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40a99b0 tx=0x7fe2c4000bc0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 shutdown_connections 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55cdc235fdd0 0x55cdc23621f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55cdc22e0430 0x55cdc22f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cdc22db850 0x55cdc2358090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 --2- 172.21.15.90:0/4163704663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cdc22dc170 0x55cdc23585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2b6a00700 0 client.15324 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 >> 172.21.15.90:0/4163704663 conn(0x55cdc2205be0 msgr2=0x55cdc22d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 shutdown_connections 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 1 -- 172.21.15.90:0/4163704663 wait complete. 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.180+0000 7fe2e1d50b80 20 client.15324 trim_cache size 0 max 16384 2024-06-02T15:44:51.186 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 msgr2=0x7fe2b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 0x7fe2b80b4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009790 tx=0x7fe2c800b330 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 msgr2=0x7fe2d019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400ec10 tx=0x7fe2c400ef20 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 shutdown_connections 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe2b80b2900 0x7fe2b80b4dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2d01088f0 0x7fe2d0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2d01037f0 0x7fe2d019f7d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 --2- 172.21.15.90:0/43346518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2d01041a0 0x7fe2d019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.181+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 >> 172.21.15.90:0/43346518 conn(0x7fe2d007a6a0 msgr2=0x7fe2d01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.182+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 shutdown_connections 2024-06-02T15:44:51.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:51.182+0000 7fe2d7400700 1 -- 172.21.15.90:0/43346518 wait complete. 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 -- 172.21.15.90:0/1002523005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0894103560 msgr2=0x7f0894105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 --2- 172.21.15.90:0/1002523005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0894103560 0x7f0894105950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0888009900 tx=0x7f088802a410 comp rx=0 tx=0).stop 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 -- 172.21.15.90:0/1002523005 shutdown_connections 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 --2- 172.21.15.90:0/1002523005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08941087c0 0x7f089410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 --2- 172.21.15.90:0/1002523005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894105e90 0x7f0894108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 --2- 172.21.15.90:0/1002523005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0894103560 0x7f0894105950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 -- 172.21.15.90:0/1002523005 >> 172.21.15.90:0/1002523005 conn(0x7f08940fcf50 msgr2=0x7f08940ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 -- 172.21.15.90:0/1002523005 shutdown_connections 2024-06-02T15:44:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.461+0000 7f089ba00640 1 -- 172.21.15.90:0/1002523005 wait complete. 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.462+0000 7f089ba00640 1 Processor -- start 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 -- start start 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0894105e90 0x7f089419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 0x7f08941a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f089410e660 con 0x7f08941087c0 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f089410e4e0 con 0x7f0894105e90 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f089ba00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f089410e7e0 con 0x7f0894103560 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f0899000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f0899a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 0x7f08941a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f0899000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38320/0 (socket says 172.21.15.90:38320) 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f0899000640 1 -- 172.21.15.90:0/1433071630 learned_addr learned my addr 172.21.15.90:0/1433071630 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.463+0000 7f0893e00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0894105e90 0x7f089419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 -- 172.21.15.90:0/1433071630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0894105e90 msgr2=0x7f089419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0894105e90 0x7f089419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 -- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 msgr2=0x7f08941a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 0x7f08941a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 -- 172.21.15.90:0/1433071630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08941a2ab0 con 0x7f0894103560 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899a00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 0x7f08941a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:51.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0899000640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0888005ab0 tx=0x7f0888003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0891600640 1 -- 172.21.15.90:0/1433071630 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088803f070 con 0x7f0894103560 2024-06-02T15:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0891600640 1 -- 172.21.15.90:0/1433071630 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0888031c50 con 0x7f0894103560 2024-06-02T15:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08941a2d40 con 0x7f0894103560 2024-06-02T15:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.464+0000 7f0891600640 1 -- 172.21.15.90:0/1433071630 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888035030 con 0x7f0894103560 2024-06-02T15:44:51.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.465+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08941a31d0 con 0x7f0894103560 2024-06-02T15:44:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.468+0000 7f0891600640 1 -- 172.21.15.90:0/1433071630 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0888042070 con 0x7f0894103560 2024-06-02T15:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.469+0000 7f089e78d740 1 Processor -- start 2024-06-02T15:44:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.469+0000 7f089e78d740 1 -- start start 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 0x562543571f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543579b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5625434e1b90 con 0x562543577690 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5625434d1730 con 0x562543571b30 2024-06-02T15:44:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f089e78d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5625434d5770 con 0x562543572520 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0891600640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 0x7f087c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0899a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 0x562543571f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0891600640 1 -- 172.21.15.90:0/1433071630 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f0888102c10 con 0x7f0894103560 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0893e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543579b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0899000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0893e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543579b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58740/0 (socket says 172.21.15.90:58740) 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0893e00640 1 -- 172.21.15.90:0/2917746043 learned_addr learned my addr 172.21.15.90:0/2917746043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0899000640 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38334/0 (socket says 172.21.15.90:38334) 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0899a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 0x562543571f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37396/0 (socket says 172.21.15.90:37396) 2024-06-02T15:44:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.470+0000 7f0899a00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 0x7f087c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 -- 172.21.15.90:0/2917746043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 msgr2=0x562543571f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 0x562543571f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 -- 172.21.15.90:0/2917746043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 msgr2=0x562543579b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543579b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 -- 172.21.15.90:0/2917746043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254346a8b0 con 0x562543572520 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899000640 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08881006d0 tx=0x7f0888005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd347a93cc0e0ee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0899a00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 0x7f087c0b4de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x562543572d80 tx=0x7f088c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0886000640 1 -- 172.21.15.90:0/2917746043 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888049070 con 0x562543572520 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0886000640 1 -- 172.21.15.90:0/2917746043 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08880ce230 con 0x562543572520 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.471+0000 7f0886000640 1 -- 172.21.15.90:0/2917746043 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08880ca3f0 con 0x562543572520 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 -- 172.21.15.90:0/2917746043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 msgr2=0x562543577020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08881006d0 tx=0x7f0888005d50 comp rx=0 tx=0).stop 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 -- 172.21.15.90:0/2917746043 shutdown_connections 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543579b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x562543577020 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 --2- 172.21.15.90:0/2917746043 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543571b30 0x562543571f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 -- 172.21.15.90:0/2917746043 >> 172.21.15.90:0/2917746043 conn(0x56254356d1a0 msgr2=0x56254356d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 -- 172.21.15.90:0/2917746043 shutdown_connections 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.472+0000 7f089e78d740 1 -- 172.21.15.90:0/2917746043 wait complete. 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 -- start start 2024-06-02T15:44:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 Processor -- start 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x56254357bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543577690 0x562543582bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5625431b7790 con 0x562543571b30 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562543479bc0 con 0x562543577690 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56254357a400 con 0x562543572520 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.473+0000 7f089e78d740 10 client.0 _ll_register_callbacks cb 0x562543457400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58746/0 (socket says 172.21.15.90:58746) 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 -- 172.21.15.90:0/1013023733 learned_addr learned my addr 172.21.15.90:0/1013023733 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0893e00640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x56254357bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899a00640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543577690 0x562543582bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 -- 172.21.15.90:0/1013023733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 msgr2=0x56254357bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x56254357bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 -- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543577690 msgr2=0x562543582bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543577690 0x562543582bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 -- 172.21.15.90:0/1013023733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625431e6210 con 0x562543571b30 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.474+0000 7f0899000640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0888005c40 tx=0x7f0888100300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b5fc785cd0717b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f087aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08880e8070 con 0x562543571b30 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f08880e9260 con 0x562543571b30 2024-06-02T15:44:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08880ce230 con 0x562543571b30 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0888049030 con 0x562543571b30 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f089e78d740 10 client.15336 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.475+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625432b2620 con 0x562543571b30 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08880ce230 con 0x562543571b30 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f08880d2850 con 0x562543571b30 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 20 client.15336 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562543586de0 0x5625435891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 4 client.15336 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625434e3ae0 con 0x562543586de0 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.476+0000 7f0893e00640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562543586de0 0x5625435891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.477+0000 7f0893e00640 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562543586de0 0x5625435891d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62fe5008c0a2b4d server_cookie=28808ff17ed2dd27 in_seq=0 out_seq=0 2024-06-02T15:44:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.477+0000 7f087aa00640 10 client.15336 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f087aa00640 1 -- 172.21.15.90:0/1013023733 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5625434e3ae0 con 0x562543586de0 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f087aa00640 10 client.15336 handle_command_reply: tid=0 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 client.15336 shutdown 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 msgr2=0x56254357b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0888005c40 tx=0x7f0888100300 comp rx=0 tx=0).stop 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 shutdown_connections 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562543586de0 0x5625435891d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543577690 0x562543582bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543572520 0x56254357bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 --2- 172.21.15.90:0/1013023733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357b640 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 >> 172.21.15.90:0/1013023733 conn(0x56254356d1a0 msgr2=0x56254356c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.478+0000 7f087aa00640 0 client.15336 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.479+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 shutdown_connections 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.479+0000 7f089e78d740 1 -- 172.21.15.90:0/1013023733 wait complete. 2024-06-02T15:44:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.479+0000 7f089e78d740 20 client.15336 trim_cache size 0 max 16384 2024-06-02T15:44:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.509+0000 7f089e78d740 1 Processor -- start 2024-06-02T15:44:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.509+0000 7f089e78d740 1 -- start start 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543571b30 0x56254357b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543572db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08941affd0 con 0x562543577690 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ad8a0 con 0x562543572520 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f089e78d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ab6f0 con 0x562543571b30 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f0893e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f0893e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37406/0 (socket says 172.21.15.90:37406) 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f0893e00640 1 -- 172.21.15.90:0/3406351034 learned_addr learned my addr 172.21.15.90:0/3406351034 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f0899000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543571b30 0x56254357b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.510+0000 7f0899a00640 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543572db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 -- 172.21.15.90:0/3406351034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543571b30 msgr2=0x56254357b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543571b30 0x56254357b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 -- 172.21.15.90:0/3406351034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 msgr2=0x562543572db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543572db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 -- 172.21.15.90:0/3406351034 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562543589800 con 0x562543572520 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0899a00640 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543572db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0893e00640 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0880009e50 tx=0x7f0880005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eef20e84ef6de17 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.511+0000 7f0885600640 1 -- 172.21.15.90:0/3406351034 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880010430 con 0x562543572520 2024-06-02T15:44:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f0885600640 1 -- 172.21.15.90:0/3406351034 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0880010a80 con 0x562543572520 2024-06-02T15:44:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f0885600640 1 -- 172.21.15.90:0/3406351034 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088001ea80 con 0x562543572520 2024-06-02T15:44:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 -- 172.21.15.90:0/3406351034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 msgr2=0x562543582b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0880009e50 tx=0x7f0880005f90 comp rx=0 tx=0).stop 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 -- 172.21.15.90:0/3406351034 shutdown_connections 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543577690 0x562543572db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x562543582b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 --2- 172.21.15.90:0/3406351034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543571b30 0x56254357b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.512+0000 7f089e78d740 1 -- 172.21.15.90:0/3406351034 >> 172.21.15.90:0/3406351034 conn(0x56254356d1a0 msgr2=0x56254357b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.513+0000 7f089e78d740 1 -- 172.21.15.90:0/3406351034 shutdown_connections 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.513+0000 7f089e78d740 1 -- 172.21.15.90:0/3406351034 wait complete. 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.513+0000 7f089e78d740 1 -- start start 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.513+0000 7f089e78d740 1 Processor -- start 2024-06-02T15:44:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x56254357ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543577690 0x5625435f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08941abd60 con 0x562543571b30 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ad1d0 con 0x562543572520 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ac430 con 0x562543577690 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f089e78d740 10 client.0 _ll_register_callbacks cb 0x7f08800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58774/0 (socket says 172.21.15.90:58774) 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899000640 1 -- 172.21.15.90:0/2669957920 learned_addr learned my addr 172.21.15.90:0/2669957920 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0893e00640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x56254357ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899a00640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543577690 0x5625435f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899000640 1 -- 172.21.15.90:0/2669957920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543577690 msgr2=0x5625435f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.514+0000 7f0899000640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543577690 0x5625435f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f0899000640 1 -- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 msgr2=0x56254357ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f0899000640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x56254357ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f0899000640 1 -- 172.21.15.90:0/2669957920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625432b2620 con 0x562543571b30 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f0899000640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f08880ca590 tx=0x7f08880c9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad1687768fdb634 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f087aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08880e8040 con 0x562543571b30 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f08880e9260 con 0x562543571b30 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0880010a80 con 0x562543571b30 2024-06-02T15:44:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08880ce230 con 0x562543571b30 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f089e78d740 10 client.15342 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.515+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56254356cfd0 con 0x562543571b30 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0880010a80 con 0x562543571b30 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f08880c9190 con 0x562543571b30 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 10 client.15342 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 10 client.15342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 20 client.15342 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5625435f9ba0 0x5625435fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 4 client.15342 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625434dd2c0 con 0x5625435f9ba0 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.516+0000 7f0893e00640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5625435f9ba0 0x5625435fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.517+0000 7f087aa00640 10 client.15342 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.517+0000 7f0893e00640 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5625435f9ba0 0x5625435fbfc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d51417495fbcc2cd server_cookie=2aa4fd02c12a1943 in_seq=0 out_seq=0 2024-06-02T15:44:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f087aa00640 1 -- 172.21.15.90:0/2669957920 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5625434dd2c0 con 0x5625435f9ba0 2024-06-02T15:44:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f087aa00640 10 client.15342 handle_command_reply: tid=0 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 client.15342 shutdown 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 msgr2=0x56254357fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f08880ca590 tx=0x7f08880c9b60 comp rx=0 tx=0).stop 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 shutdown_connections 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5625435f9ba0 0x5625435fbfc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562543577690 0x5625435f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562543572520 0x56254357ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 --2- 172.21.15.90:0/2669957920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562543571b30 0x56254357fa70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 >> 172.21.15.90:0/2669957920 conn(0x56254356d1a0 msgr2=0x56254356ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f087aa00640 0 client.15342 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 shutdown_connections 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 1 -- 172.21.15.90:0/2669957920 wait complete. 2024-06-02T15:44:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.521+0000 7f089e78d740 20 client.15342 trim_cache size 0 max 16384 2024-06-02T15:44:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:44:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:44:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 msgr2=0x7f087c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 0x7f087c0b4de0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x562543572d80 tx=0x7f088c009500 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 msgr2=0x7f089419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0888005ab0 tx=0x7f0888003b90 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 shutdown_connections 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f087c0b2920 0x7f087c0b4de0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08941087c0 0x7f08941a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0894105e90 0x7f089419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 --2- 172.21.15.90:0/1433071630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0894103560 0x7f089419ae90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 >> 172.21.15.90:0/1433071630 conn(0x7f08940fcf50 msgr2=0x7f0894102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 shutdown_connections 2024-06-02T15:44:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:44:51.522+0000 7f089ba00640 1 -- 172.21.15.90:0/1433071630 wait complete. 2024-06-02T15:44:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:51 smithi120 ceph-mon[36451]: pgmap v526: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:51 smithi135 ceph-mon[36839]: pgmap v526: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:51.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:51 smithi090 ceph-mon[31595]: pgmap v526: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-02T15:44:52.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:44:52.590 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:53.220 INFO:tasks.workunit.client.0.smithi090.stderr:Updating files: 56% (7139/12565) Updating files: 57% (7163/12565) Updating files: 58% (7288/12565) Updating files: 59% (7414/12565) Updating files: 60% (7539/12565) Updating files: 61% (7665/12565) Updating files: 62% (7791/12565) Updating files: 63% (7916/12565) Updating files: 64% (8042/12565) Updating files: 65% (8168/12565) Updating files: 66% (8293/12565) Updating files: 67% (8419/12565) Updating files: 68% (8545/12565) Updating files: 69% (8670/12565) Updating files: 70% (8796/12565) Updating files: 71% (8922/12565) Updating files: 72% (9047/12565) Updating files: 73% (9173/12565) Updating files: 74% (9299/12565) Updating files: 75% (9424/12565) Updating files: 76% (9550/12565) Updating files: 77% (9676/12565) Updating files: 78% (9801/12565) Updating files: 79% (9927/12565) Updating files: 80% (10052/12565) Updating files: 81% (10178/12565) Updating files: 82% (10304/12565) Updating files: 83% (10429/12565) Updating files: 84% (10555/12565) Updating files: 85% (10681/12565) Updating files: 86% (10806/12565) Updating files: 87% (10932/12565) Updating files: 88% (11058/12565) Updating files: 89% (11183/12565) Updating files: 90% (11309/12565) Updating files: 90% (11326/12565) Updating files: 91% (11435/12565) Updating files: 92% (11560/12565) Updating files: 93% (11686/12565) Updating files: 94% (11812/12565) Updating files: 95% (11937/12565) Updating files: 96% (12063/12565) Updating files: 97% (12189/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-02T15:44:53.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:53 smithi090 ceph-mon[31595]: pgmap v527: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:44:53.492 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '7aec00165b4b073a25396fd0279151d2b2fa23fe'. 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.493 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-06-02T15:44:53.494 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 7aec00165b4 Merge pull request #57616 from Matan-B/wip-matanb-fix-python-tests 2024-06-02T15:44:53.497 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-02T15:44:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:53 smithi120 ceph-mon[36451]: pgmap v527: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:44:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:53 smithi135 ceph-mon[36839]: pgmap v527: 129 pgs: 129 active+clean; 670 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 341 B/s wr, 5 op/s 2024-06-02T15:44:53.620 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-02T15:44:53.629 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-02T15:44:53.629 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-02T15:44:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 -- 172.21.15.90:0/10044966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 msgr2=0x7f7f74109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 --2- 172.21.15.90:0/10044966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f74109900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b0d0 tx=0x7f7f6400b3e0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 -- 172.21.15.90:0/10044966 shutdown_connections 2024-06-02T15:44:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 --2- 172.21.15.90:0/10044966 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f741024f0 0x7f7f741103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 --2- 172.21.15.90:0/10044966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f74109900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 --2- 172.21.15.90:0/10044966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f74101b30 0x7f7f74101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 -- 172.21.15.90:0/10044966 >> 172.21.15.90:0/10044966 conn(0x7f7f74078ab0 msgr2=0x7f7f740ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 -- 172.21.15.90:0/10044966 shutdown_connections 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.218+0000 7f7f7a400700 1 -- 172.21.15.90:0/10044966 wait complete. 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.219+0000 7f7f7a400700 1 Processor -- start 2024-06-02T15:44:55.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.219+0000 7f7f7a400700 1 -- start start 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.219+0000 7f7f7a400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 0x7f7f7419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.219+0000 7f7f7a400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f741024f0 0x7f7f7419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f7a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f7a400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f74113770 con 0x7f7f74109520 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f7a400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741135f0 con 0x7f7f74101b30 2024-06-02T15:44:55.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f7a400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741138f0 con 0x7f7f741024f0 2024-06-02T15:44:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58784/0 (socket says 172.21.15.90:58784) 2024-06-02T15:44:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f72a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f741024f0 0x7f7f7419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 -- 172.21.15.90:0/4183295938 learned_addr learned my addr 172.21.15.90:0/4183295938 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 0x7f7f7419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 -- 172.21.15.90:0/4183295938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f741024f0 msgr2=0x7f7f7419c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f741024f0 0x7f7f7419c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 -- 172.21.15.90:0/4183295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 msgr2=0x7f7f7419be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 0x7f7f7419be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73e00700 1 -- 172.21.15.90:0/4183295938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f741a0e90 con 0x7f7f74109520 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.220+0000 7f7f73400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 0x7f7f7419be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f73e00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e940 tx=0x7f7f6c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f6be00700 1 -- 172.21.15.90:0/4183295938 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004a30 con 0x7f7f74109520 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f6be00700 1 -- 172.21.15.90:0/4183295938 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f6c0054d0 con 0x7f7f74109520 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f6be00700 1 -- 172.21.15.90:0/4183295938 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00ccf0 con 0x7f7f74109520 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f741a1180 con 0x7f7f74109520 2024-06-02T15:44:55.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.221+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f741a8a00 con 0x7f7f74109520 2024-06-02T15:44:55.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.225+0000 7f7f6be00700 1 -- 172.21.15.90:0/4183295938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f6c005020 con 0x7f7f74109520 2024-06-02T15:44:55.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.226+0000 7f7f84df7b80 1 Processor -- start 2024-06-02T15:44:55.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.226+0000 7f7f84df7b80 1 -- start start 2024-06-02T15:44:55.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f5997959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f5997905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f5996ede40 con 0x55f599795610 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f5996dd9e0 con 0x55f599790140 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f5996e1a20 con 0x55f599795f30 2024-06-02T15:44:55.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f6be00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 0x7f7f580b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f73400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f6be00700 1 -- 172.21.15.90:0/4183295938 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f6c0e26a0 con 0x7f7f74109520 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f73400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38386/0 (socket says 172.21.15.90:38386) 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f73400700 1 -- 172.21.15.90:0/3004316415 learned_addr learned my addr 172.21.15.90:0/3004316415 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f73e00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 0x7f7f580b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f5997959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f5997905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.227+0000 7f7f73400700 1 -- 172.21.15.90:0/3004316415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 msgr2=0x55f5997905c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73400700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f5997905c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73400700 1 -- 172.21.15.90:0/3004316415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 msgr2=0x55f5997959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73400700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f5997959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73400700 1 -- 172.21.15.90:0/3004316415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5996a6420 con 0x55f599795f30 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f5997905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73e00700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 0x7f7f580b4dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0a5930 tx=0x7f7f6c004f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.228+0000 7f7f73400700 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009c00 tx=0x7f7f64009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf75b8fa35b2cb8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.229+0000 7f7f68c00700 1 -- 172.21.15.90:0/3004316415 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64004a50 con 0x55f599795f30 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.229+0000 7f7f68c00700 1 -- 172.21.15.90:0/3004316415 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f64004bf0 con 0x55f599795f30 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.229+0000 7f7f68c00700 1 -- 172.21.15.90:0/3004316415 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640052a0 con 0x55f599795f30 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.229+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3004316415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 msgr2=0x55f59978fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.229+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009c00 tx=0x7f7f64009c30 comp rx=0 tx=0).stop 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3004316415 shutdown_connections 2024-06-02T15:44:55.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f5997905c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f5997959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3004316415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f59978fb20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3004316415 >> 172.21.15.90:0/3004316415 conn(0x55f5996bdbe0 msgr2=0x55f59978a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3004316415 shutdown_connections 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3004316415 wait complete. 2024-06-02T15:44:55.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.230+0000 7f7f84df7b80 1 -- start start 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 Processor -- start 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f59979bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f599799450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f5995e3fa0 con 0x55f599795610 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f59967e760 con 0x55f599790140 2024-06-02T15:44:55.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f59979c980 con 0x55f599795f30 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f84df7b80 10 client.0 _ll_register_callbacks cb 0x55f599687c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f73400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58790/0 (socket says 172.21.15.90:58790) 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.231+0000 7f7f73400700 1 -- 172.21.15.90:0/3049949938 learned_addr learned my addr 172.21.15.90:0/3049949938 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f59979bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f599799450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 -- 172.21.15.90:0/3049949938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 msgr2=0x55f599799450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f599799450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 -- 172.21.15.90:0/3049949938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 msgr2=0x55f59979bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f59979bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 -- 172.21.15.90:0/3049949938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f59962d8d0 con 0x55f599795610 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f73400700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b770 tx=0x7f7f64007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae889ca05bbf8b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.232+0000 7f7f5d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:44:55.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64033060 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f6402a040 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f64004bf0 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f6403a050 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f84df7b80 10 client.15354 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5996ae060 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f64004bf0 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.233+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7f64011070 con 0x55f599795610 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 10 client.15354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 20 client.15354 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f5997a9290 0x55f5997ab680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 4 client.15354 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:44:55.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.234+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5996efd90 con 0x55f5997a9290 2024-06-02T15:44:55.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.235+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f5997a9290 0x55f5997ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.235+0000 7f7f72a00700 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f5997a9290 0x55f5997ab680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7537850b4b23184e server_cookie=a900eb25529aa23d in_seq=0 out_seq=0 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.235+0000 7f7f5d600700 10 client.15354 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f5d600700 1 -- 172.21.15.90:0/3049949938 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f5996efd90 con 0x55f5997a9290 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f5d600700 10 client.15354 handle_command_reply: tid=0 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 client.15354 shutdown 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 msgr2=0x55f599798f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b770 tx=0x7f7f64007d10 comp rx=0 tx=0).stop 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 shutdown_connections 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f599799450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f5997a9290 0x55f5997ab680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599790140 0x55f59979bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f5d600700 0 client.15354 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:55.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/3049949938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599795610 0x55f599798f10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.236+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 >> 172.21.15.90:0/3049949938 conn(0x55f5996bdbe0 msgr2=0x55f599789d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.237+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 shutdown_connections 2024-06-02T15:44:55.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.237+0000 7f7f84df7b80 1 -- 172.21.15.90:0/3049949938 wait complete. 2024-06-02T15:44:55.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.237+0000 7f7f84df7b80 20 client.15354 trim_cache size 0 max 16384 2024-06-02T15:44:55.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.253+0000 7f7f84df7b80 1 Processor -- start 2024-06-02T15:44:55.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 -- start start 2024-06-02T15:44:55.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f59978a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b3db0 con 0x55f599790140 2024-06-02T15:44:55.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b1640 con 0x55f599795610 2024-06-02T15:44:55.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741af490 con 0x55f599795f30 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58796/0 (socket says 172.21.15.90:58796) 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f72a00700 1 -- 172.21.15.90:0/2774493824 learned_addr learned my addr 172.21.15.90:0/2774493824 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.254+0000 7f7f73400700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f59978a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 -- 172.21.15.90:0/2774493824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 msgr2=0x55f5997ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 -- 172.21.15.90:0/2774493824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 msgr2=0x55f59978a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f59978a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 -- 172.21.15.90:0/2774493824 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5997939b0 con 0x55f599790140 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f73400700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f59978a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f72a00700 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55f599790b00 tx=0x7f7f60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aac4afa602e461c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f5fe00700 1 -- 172.21.15.90:0/2774493824 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600114e0 con 0x55f599790140 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f5fe00700 1 -- 172.21.15.90:0/2774493824 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f600044e0 con 0x55f599790140 2024-06-02T15:44:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.255+0000 7f7f5fe00700 1 -- 172.21.15.90:0/2774493824 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600038e0 con 0x55f599790140 2024-06-02T15:44:55.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.256+0000 7f7f84df7b80 1 -- 172.21.15.90:0/2774493824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 msgr2=0x55f59978fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.256+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55f599790b00 tx=0x7f7f60004750 comp rx=0 tx=0).stop 2024-06-02T15:44:55.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 -- 172.21.15.90:0/2774493824 shutdown_connections 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f59978fb20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/2774493824 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f59978a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 -- 172.21.15.90:0/2774493824 >> 172.21.15.90:0/2774493824 conn(0x55f5996bdbe0 msgr2=0x55f599798ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 -- 172.21.15.90:0/2774493824 shutdown_connections 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 -- 172.21.15.90:0/2774493824 wait complete. 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 -- start start 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.257+0000 7f7f84df7b80 1 Processor -- start 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f5997aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f5997abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741afb00 con 0x55f599790140 2024-06-02T15:44:55.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b0f70 con 0x55f599795610 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b01d0 con 0x55f599795f30 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f84df7b80 10 client.0 _ll_register_callbacks cb 0x55f599687c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f5997aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f72a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f5997aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58802/0 (socket says 172.21.15.90:58802) 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f72a00700 1 -- 172.21.15.90:0/500740148 learned_addr learned my addr 172.21.15.90:0/500740148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.258+0000 7f7f73400700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f5997abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f72a00700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 -- 172.21.15.90:0/500740148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 msgr2=0x55f5997ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ac400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 -- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 msgr2=0x55f5997aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f5997aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 -- 172.21.15.90:0/500740148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5997ac9c0 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f73400700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f5997abec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004280 tx=0x7f7f64005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc3db6b87e937a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.259+0000 7f7f5d600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64033040 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f6402a040 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f600044e0 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7f64047050 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f84df7b80 10 client.25496 fetch_fsmap learned FSMap version 26 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5998138a0 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.260+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f600044e0 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7f64011070 con 0x55f599795610 2024-06-02T15:44:55.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 10 client.25496 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 10 client.25496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 20 client.25496 populate_metadata read hostname 'smithi090' 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f599813ef0 0x55f5998162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 4 client.25496 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5996e9570 con 0x55f599813ef0 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.261+0000 7f7f72a00700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f599813ef0 0x55f5998162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f72a00700 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f599813ef0 0x55f5998162e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba48db44a1467d38 server_cookie=19c915a7da3596aa in_seq=0 out_seq=0 2024-06-02T15:44:55.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f5d600700 10 client.25496 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f5d600700 1 -- 172.21.15.90:0/500740148 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5996e9570 con 0x55f599813ef0 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f5d600700 10 client.25496 handle_command_reply: tid=0 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 client.25496 shutdown 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 msgr2=0x55f5997abec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f5997abec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004280 tx=0x7f7f64005e90 comp rx=0 tx=0).stop 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 shutdown_connections 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f599795f30 0x55f5997ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f599790140 0x55f5997aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f599813ef0 0x55f5998162e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 --2- 172.21.15.90:0/500740148 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f599795610 0x55f5997abec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f5d600700 0 client.25496 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 >> 172.21.15.90:0/500740148 conn(0x55f5996bdbe0 msgr2=0x55f599797820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.262+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 shutdown_connections 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f84df7b80 1 -- 172.21.15.90:0/500740148 wait complete. 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f84df7b80 20 client.25496 trim_cache size 0 max 16384 2024-06-02T15:44:55.263 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:44:55.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 msgr2=0x7f7f580b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 0x7f7f580b4dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0a5930 tx=0x7f7f6c004f30 comp rx=0 tx=0).stop 2024-06-02T15:44:55.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 msgr2=0x7f7f741a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:44:55.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e940 tx=0x7f7f6c00ec50 comp rx=0 tx=0).stop 2024-06-02T15:44:55.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 shutdown_connections 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f580b2900 0x7f7f580b4dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f74109520 0x7f7f741a0750 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f74101b30 0x7f7f7419be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 --2- 172.21.15.90:0/4183295938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f741024f0 0x7f7f7419c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.263+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 >> 172.21.15.90:0/4183295938 conn(0x7f7f74078ab0 msgr2=0x7f7f74100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.264+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 shutdown_connections 2024-06-02T15:44:55.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:44:55.264+0000 7f7f7a400700 1 -- 172.21.15.90:0/4183295938 wait complete. 2024-06-02T15:44:55.543 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-02T15:44:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:55 smithi120 ceph-mon[36451]: pgmap v528: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-02T15:44:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:55 smithi135 ceph-mon[36839]: pgmap v528: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-02T15:44:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:55 smithi090 ceph-mon[31595]: pgmap v528: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-06-02T15:44:55.682 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-02T15:44:55.761 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-02T15:44:55.767 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-02T15:44:55.767 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-02T15:44:55.850 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-02T15:44:55.860 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:44:55.860 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-02T15:44:55.910 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2024-06-02T15:44:55.911 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2024-06-02T15:44:55.912 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2024-06-02T15:44:56.023 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:44:56.024 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p testdir 2024-06-02T15:44:56.028 INFO:tasks.workunit.client.0.smithi090.stderr:+ cd testdir 2024-06-02T15:44:56.029 INFO:tasks.workunit.client.0.smithi090.stderr:+ set +e 2024-06-02T15:44:56.029 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfacl -d -m u:nobody:rw . 2024-06-02T15:44:56.050 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:44:56.050 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:44:56.050 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=0 2024-06-02T15:44:56.050 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 0 -lt 100 ']' 2024-06-02T15:44:56.051 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 0 + 1 2024-06-02T15:44:56.053 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=1 2024-06-02T15:44:56.053 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:44:56.060 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:44:56.060 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:44:56.120 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:44:56.120 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:44:56.121 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:44:56.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:44:56.954 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:44:57.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:57 smithi090 ceph-mon[31595]: pgmap v529: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:57 smithi120 ceph-mon[36451]: pgmap v529: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:57 smithi135 ceph-mon[36839]: pgmap v529: 129 pgs: 129 active+clean; 671 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-06-02T15:44:59.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:44:59 smithi090 ceph-mon[31595]: pgmap v530: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:44:59.491 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:44:59.492 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:44:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:44:59 smithi120 ceph-mon[36451]: pgmap v530: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:44:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:44:59 smithi135 ceph-mon[36839]: pgmap v530: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:44:59.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:44:59.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:44:59.683 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:44:59.683 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:44:59.683 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 1 -lt 100 ']' 2024-06-02T15:44:59.684 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 1 + 1 2024-06-02T15:44:59.730 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=2 2024-06-02T15:44:59.731 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:44:59.783 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:44:59.783 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:44:59.787 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:44:59.788 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:44:59.788 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:00.899 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:00.899 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:01.291 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:01.291 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:01.319 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:01.319 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:01.319 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -lt 100 ']' 2024-06-02T15:45:01.321 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 2 + 1 2024-06-02T15:45:01.370 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=3 2024-06-02T15:45:01.371 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:01.393 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:01.393 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:01.396 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:01.397 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:01.397 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:01 smithi120 ceph-mon[36451]: pgmap v531: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:45:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:01 smithi135 ceph-mon[36839]: pgmap v531: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:45:01.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:01 smithi090 ceph-mon[31595]: pgmap v531: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-06-02T15:45:02.915 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:02.916 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:03.374 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:03.374 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:03.413 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:03.413 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:03.414 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 3 -lt 100 ']' 2024-06-02T15:45:03.414 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 3 + 1 2024-06-02T15:45:03.468 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=4 2024-06-02T15:45:03.468 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:03.531 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:03.532 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:03.536 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:03.537 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:03.538 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:03 smithi120 ceph-mon[36451]: pgmap v532: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-02T15:45:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:03 smithi135 ceph-mon[36839]: pgmap v532: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-02T15:45:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:03 smithi090 ceph-mon[31595]: pgmap v532: 129 pgs: 129 active+clean; 685 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-06-02T15:45:05.157 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:05.157 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:05 smithi135 ceph-mon[36839]: pgmap v533: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-02T15:45:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:05 smithi090 ceph-mon[31595]: pgmap v533: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-02T15:45:05.757 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:05.757 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:05.794 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:05.794 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:05.794 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 4 -lt 100 ']' 2024-06-02T15:45:05.795 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 4 + 1 2024-06-02T15:45:05.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:05 smithi120 ceph-mon[36451]: pgmap v533: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-06-02T15:45:05.918 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=5 2024-06-02T15:45:05.919 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:06.031 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:06.031 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:06.035 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:06.036 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:06.037 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 -- 172.21.15.90:0/2875866111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 msgr2=0x7f66a8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 --2- 172.21.15.90:0/2875866111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a8112650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009f90 tx=0x7f66a000b330 comp rx=0 tx=0).stop 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 -- 172.21.15.90:0/2875866111 shutdown_connections 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 --2- 172.21.15.90:0/2875866111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a8112650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 --2- 172.21.15.90:0/2875866111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a810b780 0x7f66a810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 --2- 172.21.15.90:0/2875866111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a8105780 0x7f66a8105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 -- 172.21.15.90:0/2875866111 >> 172.21.15.90:0/2875866111 conn(0x7f66a8101270 msgr2=0x7f66a8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 -- 172.21.15.90:0/2875866111 shutdown_connections 2024-06-02T15:45:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.506+0000 7f66aea00700 1 -- 172.21.15.90:0/2875866111 wait complete. 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.507+0000 7f66aea00700 1 Processor -- start 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 -- start start 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 0x7f66a81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81159b0 con 0x7f66a8105780 2024-06-02T15:45:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8115830 con 0x7f66a8106140 2024-06-02T15:45:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.508+0000 7f66aea00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a8115b30 con 0x7f66a810b780 2024-06-02T15:45:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 0x7f66a81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49462/0 (socket says 172.21.15.90:49462) 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66a7e00700 1 -- 172.21.15.90:0/2319869996 learned_addr learned my addr 172.21.15.90:0/2319869996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66a7400700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 -- 172.21.15.90:0/2319869996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 msgr2=0x7f66a819c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a819c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 -- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 msgr2=0x7f66a819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66aca00700 1 -- 172.21.15.90:0/2319869996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a81a0dd0 con 0x7f66a810b780 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.509+0000 7f66a7e00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66aca00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 0x7f66a81a0690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b6c0 tx=0x7f66a000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66a4c00700 1 -- 172.21.15.90:0/2319869996 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0021070 con 0x7f66a810b780 2024-06-02T15:45:06.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66a4c00700 1 -- 172.21.15.90:0/2319869996 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0007d60 con 0x7f66a810b780 2024-06-02T15:45:06.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66a4c00700 1 -- 172.21.15.90:0/2319869996 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0005040 con 0x7f66a810b780 2024-06-02T15:45:06.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81a1060 con 0x7f66a810b780 2024-06-02T15:45:06.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.510+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81a8940 con 0x7f66a810b780 2024-06-02T15:45:06.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.513+0000 7f66a4c00700 1 -- 172.21.15.90:0/2319869996 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f66a0024070 con 0x7f66a810b780 2024-06-02T15:45:06.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.515+0000 7f66a4c00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 0x7f668c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.515+0000 7f66a4c00700 1 -- 172.21.15.90:0/2319869996 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f66a00e35c0 con 0x7f66a810b780 2024-06-02T15:45:06.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.515+0000 7f66a7e00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 0x7f668c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.516+0000 7f66a7e00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 0x7f668c0b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6698009750 tx=0x7f6698009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.628+0000 7f66b9304b80 1 Processor -- start 2024-06-02T15:45:06.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.628+0000 7f66b9304b80 1 -- start start 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.629+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.629+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7ef720 0x557e6e7e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66b9304b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557e6e73ee40 con 0x557e6e7ef720 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66b9304b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e6e72e9e0 con 0x557e6e7e9950 2024-06-02T15:45:06.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66b9304b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e6e732a20 con 0x557e6e7eee00 2024-06-02T15:45:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66aca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7ef720 0x557e6e7e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58424/0 (socket says 172.21.15.90:58424) 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66aca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:44306/0 (socket says 172.21.15.90:44306) 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.630+0000 7f66aca00700 1 -- 172.21.15.90:0/1594816798 learned_addr learned my addr 172.21.15.90:0/1594816798 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 -- 172.21.15.90:0/1594816798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 msgr2=0x557e6e7ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 -- 172.21.15.90:0/1594816798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7ef720 msgr2=0x557e6e7e9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7ef720 0x557e6e7e9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 -- 172.21.15.90:0/1594816798 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6e6f7420 con 0x557e6e7e9950 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66a7e00700 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:06.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f66aca00700 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f66a00aa7f0 tx=0x7f66a0006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1851f993fa693b61 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f669d600700 1 -- 172.21.15.90:0/1594816798 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00e2070 con 0x557e6e7e9950 2024-06-02T15:45:06.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f669d600700 1 -- 172.21.15.90:0/1594816798 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00aa860 con 0x557e6e7e9950 2024-06-02T15:45:06.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.631+0000 7f669d600700 1 -- 172.21.15.90:0/1594816798 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a002b070 con 0x557e6e7e9950 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 -- 172.21.15.90:0/1594816798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 msgr2=0x557e6e7e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f66a00aa7f0 tx=0x7f66a0006000 comp rx=0 tx=0).stop 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 -- 172.21.15.90:0/1594816798 shutdown_connections 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7e9db0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 --2- 172.21.15.90:0/1594816798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7ef720 0x557e6e7e9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 -- 172.21.15.90:0/1594816798 >> 172.21.15.90:0/1594816798 conn(0x557e6e70ebe0 msgr2=0x557e6e7db690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 -- 172.21.15.90:0/1594816798 shutdown_connections 2024-06-02T15:45:06.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.633+0000 7f66b9304b80 1 -- 172.21.15.90:0/1594816798 wait complete. 2024-06-02T15:45:06.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.634+0000 7f66b9304b80 1 -- start start 2024-06-02T15:45:06.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.634+0000 7f66b9304b80 1 Processor -- start 2024-06-02T15:45:06.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 0x557e6e7f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557e6e634fa0 con 0x557e6e7e9950 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557e6e6cf760 con 0x557e6e7ef720 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e6e7f5fe0 con 0x557e6e7eee00 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66b9304b80 10 client.0 _ll_register_callbacks cb 0x557e6e6d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49478/0 (socket says 172.21.15.90:49478) 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66a7e00700 1 -- 172.21.15.90:0/4115366180 learned_addr learned my addr 172.21.15.90:0/4115366180 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66aca00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 0x557e6e7f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.635+0000 7f66a7400700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 -- 172.21.15.90:0/4115366180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 msgr2=0x557e6e7f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 -- 172.21.15.90:0/4115366180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 msgr2=0x557e6e7f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 -- 172.21.15.90:0/4115366180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6e67e8d0 con 0x557e6e7ef720 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66aca00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 0x557e6e7f2db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66a00aa560 tx=0x7f66a00aec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b874f20fa9aef306 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f6692000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00e2070 con 0x557e6e7ef720 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f66a00a6e50 con 0x557e6e7ef720 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.636+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a00aa860 con 0x557e6e7ef720 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66a002b050 con 0x557e6e7ef720 2024-06-02T15:45:06.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f66b9304b80 10 client.25508 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:06.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e6e6ff060 con 0x557e6e7ef720 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00aa860 con 0x557e6e7ef720 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f66a0011070 con 0x557e6e7ef720 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f66b9304b80 10 client.25508 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f66b9304b80 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.637+0000 7f66b9304b80 20 client.25508 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.638+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e802970 0x557e6e804d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.638+0000 7f66b9304b80 4 client.25508 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.638+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e6e740d90 con 0x557e6e802970 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.638+0000 7f66a7e00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e802970 0x557e6e804d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.639+0000 7f66a7e00700 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e802970 0x557e6e804d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3e78115f820809 server_cookie=d924131c3a37c61 in_seq=0 out_seq=0 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.639+0000 7f6692000700 10 client.25508 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.639+0000 7f6692000700 1 -- 172.21.15.90:0/4115366180 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557e6e740d90 con 0x557e6e802970 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.639+0000 7f6692000700 10 client.25508 handle_command_reply: tid=0 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 client.25508 shutdown 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 msgr2=0x557e6e7f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 0x557e6e7f2db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66a00aa560 tx=0x7f66a00aec30 comp rx=0 tx=0).stop 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 shutdown_connections 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e802970 0x557e6e804d60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7ef720 0x557e6e7f2db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4115366180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7eee00 0x557e6e7f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 >> 172.21.15.90:0/4115366180 conn(0x557e6e70ebe0 msgr2=0x557e6e7dad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f6692000700 0 client.25508 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 shutdown_connections 2024-06-02T15:45:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.640+0000 7f66b9304b80 1 -- 172.21.15.90:0/4115366180 wait complete. 2024-06-02T15:45:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.641+0000 7f66b9304b80 20 client.25508 trim_cache size 0 max 16384 2024-06-02T15:45:06.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.658+0000 7f66b9304b80 1 Processor -- start 2024-06-02T15:45:06.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.658+0000 7f66b9304b80 1 -- start start 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81af860 con 0x557e6e7e9950 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81ad0f0 con 0x557e6e7eee00 2024-06-02T15:45:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66b9304b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81aaf40 con 0x557e6e7ef720 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66a7400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66a7400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:44328/0 (socket says 172.21.15.90:44328) 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66a7400700 1 -- 172.21.15.90:0/3279213934 learned_addr learned my addr 172.21.15.90:0/3279213934 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.659+0000 7f66aca00700 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 -- 172.21.15.90:0/3279213934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 msgr2=0x557e6e7db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 -- 172.21.15.90:0/3279213934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 msgr2=0x557e6e7ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 -- 172.21.15.90:0/3279213934 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e6e7e9580 con 0x557e6e7eee00 2024-06-02T15:45:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f66a7400700 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6694009e70 tx=0x7f6694009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f43f5f1f094e40 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.660+0000 7f669cc00700 1 -- 172.21.15.90:0/3279213934 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694022070 con 0x557e6e7eee00 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f669cc00700 1 -- 172.21.15.90:0/3279213934 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694004500 con 0x557e6e7eee00 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f669cc00700 1 -- 172.21.15.90:0/3279213934 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694003cf0 con 0x557e6e7eee00 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f66b9304b80 1 -- 172.21.15.90:0/3279213934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 msgr2=0x557e6e7f0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f66b9304b80 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6694009e70 tx=0x7f6694009260 comp rx=0 tx=0).stop 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f66b9304b80 1 -- 172.21.15.90:0/3279213934 shutdown_connections 2024-06-02T15:45:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.661+0000 7f66b9304b80 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7e9950 0x557e6e7ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 --2- 172.21.15.90:0/3279213934 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7eee00 0x557e6e7f0390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 -- 172.21.15.90:0/3279213934 >> 172.21.15.90:0/3279213934 conn(0x557e6e70ebe0 msgr2=0x557e6e7f1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 -- 172.21.15.90:0/3279213934 shutdown_connections 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 -- 172.21.15.90:0/3279213934 wait complete. 2024-06-02T15:45:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.662+0000 7f66b9304b80 1 -- start start 2024-06-02T15:45:06.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 Processor -- start 2024-06-02T15:45:06.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7eee00 0x557e6e7fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81ab5b0 con 0x557e6e7eee00 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81aca20 con 0x557e6e7e9950 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66a81abc80 con 0x557e6e7ef720 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66b9304b80 10 client.0 _ll_register_callbacks cb 0x557e6e6d8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.663+0000 7f66a7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7eee00 0x557e6e7fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:44338/0 (socket says 172.21.15.90:44338) 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 -- 172.21.15.90:0/4036617408 learned_addr learned my addr 172.21.15.90:0/4036617408 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66aca00700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 -- 172.21.15.90:0/4036617408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 msgr2=0x557e6e7f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 -- 172.21.15.90:0/4036617408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7eee00 msgr2=0x557e6e7fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7eee00 0x557e6e7fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66a7e00700 1 -- 172.21.15.90:0/4036617408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e6e7fbcb0 con 0x557e6e7e9950 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.664+0000 7f66aca00700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f66a7e00700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557e6e805510 tx=0x7f66980047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a266a2897a8b07 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:06.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f6692000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:45:06.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004220 con 0x557e6e7e9950 2024-06-02T15:45:06.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6694004500 con 0x557e6e7e9950 2024-06-02T15:45:06.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f66980043c0 con 0x557e6e7e9950 2024-06-02T15:45:06.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f669803e050 con 0x557e6e7e9950 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f66b9304b80 10 client.25514 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.665+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e6e86da20 con 0x557e6e7e9950 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694004500 con 0x557e6e7e9950 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f6698003980 con 0x557e6e7e9950 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 10 client.25514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 20 client.25514 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e86e130 0x557e6e870550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 4 client.25514 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.666+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e6e73a570 con 0x557e6e86e130 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f66a7400700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e86e130 0x557e6e870550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f6692000700 10 client.25514 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f66a7400700 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e86e130 0x557e6e870550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581a33b5600a33ef server_cookie=29c605b2e828ece4 in_seq=0 out_seq=0 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f6692000700 1 -- 172.21.15.90:0/4036617408 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e6e73a570 con 0x557e6e86e130 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f6692000700 10 client.25514 handle_command_reply: tid=0 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.667+0000 7f66b9304b80 1 client.25514 shutdown 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 msgr2=0x557e6e7fb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557e6e805510 tx=0x7f66980047c0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 shutdown_connections 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557e6e86e130 0x557e6e870550 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e6e7ef720 0x557e6e7f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557e6e7e9950 0x557e6e7fb130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f6692000700 0 client.25514 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 --2- 172.21.15.90:0/4036617408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e6e7eee00 0x557e6e7fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 >> 172.21.15.90:0/4036617408 conn(0x557e6e70ebe0 msgr2=0x557e6e7f0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 shutdown_connections 2024-06-02T15:45:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 1 -- 172.21.15.90:0/4036617408 wait complete. 2024-06-02T15:45:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.668+0000 7f66b9304b80 20 client.25514 trim_cache size 0 max 16384 2024-06-02T15:45:06.668 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:45:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 msgr2=0x7f668c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 0x7f668c0b4dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6698009750 tx=0x7f6698009310 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 msgr2=0x7f66a81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 0x7f66a81a0690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b6c0 tx=0x7f66a000cc30 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 shutdown_connections 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f668c0b2900 0x7f668c0b4dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66a810b780 0x7f66a81a0690 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66a8105780 0x7f66a819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 --2- 172.21.15.90:0/2319869996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66a8106140 0x7f66a819c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 >> 172.21.15.90:0/2319869996 conn(0x7f66a8101270 msgr2=0x7f66a81017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 shutdown_connections 2024-06-02T15:45:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:06.669+0000 7f66aea00700 1 -- 172.21.15.90:0/2319869996 wait complete. 2024-06-02T15:45:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:07 smithi135 ceph-mon[36839]: pgmap v534: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-06-02T15:45:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:07 smithi090 ceph-mon[31595]: pgmap v534: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-06-02T15:45:07.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:07 smithi120 ceph-mon[36451]: pgmap v534: 129 pgs: 129 active+clean; 715 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-06-02T15:45:08.240 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:08.240 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:08.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:08.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:08.787 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:08.787 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:08.788 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 5 -lt 100 ']' 2024-06-02T15:45:08.789 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 5 + 1 2024-06-02T15:45:08.868 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=6 2024-06-02T15:45:08.868 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:08.928 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:08.929 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:08.933 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:08.934 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:08.935 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:09.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:09 smithi090 ceph-mon[31595]: pgmap v535: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-06-02T15:45:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:09 smithi120 ceph-mon[36451]: pgmap v535: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-06-02T15:45:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:09 smithi135 ceph-mon[36839]: pgmap v535: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 7 op/s 2024-06-02T15:45:10.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.304+0000 7f3657200640 1 -- 172.21.15.90:0/15207810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 msgr2=0x7f3650106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.304+0000 7f3657200640 1 --2- 172.21.15.90:0/15207810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f3650106340 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3640009900 tx=0x7f364002a400 comp rx=0 tx=0).stop 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.304+0000 7f3657200640 1 -- 172.21.15.90:0/15207810 shutdown_connections 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.304+0000 7f3657200640 1 --2- 172.21.15.90:0/15207810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650106880 0x7f365010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.305+0000 7f3657200640 1 --2- 172.21.15.90:0/15207810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f3650106340 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.305+0000 7f3657200640 1 --2- 172.21.15.90:0/15207810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650104cd0 0x7f36501050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.305+0000 7f3657200640 1 -- 172.21.15.90:0/15207810 >> 172.21.15.90:0/15207810 conn(0x7f3650100240 msgr2=0x7f3650102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.305+0000 7f3657200640 1 -- 172.21.15.90:0/15207810 shutdown_connections 2024-06-02T15:45:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.305+0000 7f3657200640 1 -- 172.21.15.90:0/15207810 wait complete. 2024-06-02T15:45:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.411+0000 7f3657200640 1 Processor -- start 2024-06-02T15:45:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.411+0000 7f3657200640 1 -- start start 2024-06-02T15:45:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.419+0000 7f3657200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 0x7f365019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.419+0000 7f3657200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f365019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.419+0000 7f3657200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 0x7f36501a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f3657200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3650110830 con 0x7f3650106880 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f3657200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36501106b0 con 0x7f3650104cd0 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f3657200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36501109b0 con 0x7f3650105ec0 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f364fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 0x7f365019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f364f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f365019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f364f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f365019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58466/0 (socket says 172.21.15.90:58466) 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f364f400640 1 -- 172.21.15.90:0/3524662587 learned_addr learned my addr 172.21.15.90:0/3524662587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.420+0000 7f3655200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 0x7f36501a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 -- 172.21.15.90:0/3524662587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 msgr2=0x7f365019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f365019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 -- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 msgr2=0x7f36501a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:10.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 0x7f36501a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 -- 172.21.15.90:0/3524662587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36501a2a90 con 0x7f3650104cd0 2024-06-02T15:45:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f3655200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 0x7f36501a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:45:10.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.421+0000 7f364fe00640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 0x7f365019ae70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f363800e9c0 tx=0x7f363800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.427+0000 7f364cc00640 1 -- 172.21.15.90:0/3524662587 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800cd80 con 0x7f3650104cd0 2024-06-02T15:45:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.427+0000 7f364cc00640 1 -- 172.21.15.90:0/3524662587 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36380108e0 con 0x7f3650104cd0 2024-06-02T15:45:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.427+0000 7f364cc00640 1 -- 172.21.15.90:0/3524662587 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638018900 con 0x7f3650104cd0 2024-06-02T15:45:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.427+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36501a2d80 con 0x7f3650104cd0 2024-06-02T15:45:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.437+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36501a3210 con 0x7f3650104cd0 2024-06-02T15:45:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.441+0000 7f364cc00640 1 -- 172.21.15.90:0/3524662587 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f363801f020 con 0x7f3650104cd0 2024-06-02T15:45:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.445+0000 7f364cc00640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 0x7f36340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.445+0000 7f364cc00640 1 -- 172.21.15.90:0/3524662587 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f36380e5170 con 0x7f3650104cd0 2024-06-02T15:45:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.445+0000 7f364f400640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 0x7f36340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:10.446+0000 7f364f400640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 0x7f36340b4eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f365019c380 tx=0x7f3640005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:10.907 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:10.907 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.013+0000 7f365a01b740 1 Processor -- start 2024-06-02T15:45:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.014+0000 7f365a01b740 1 -- start start 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 0x55683f1b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1b1690 0x55683f1b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55683f11bb90 con 0x55683f1ac520 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55683f10b730 con 0x55683f1abb30 2024-06-02T15:45:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f365a01b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55683f10f770 con 0x55683f1b1690 2024-06-02T15:45:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f364f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 0x55683f1b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f364f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 0x55683f1b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55898/0 (socket says 172.21.15.90:55898) 2024-06-02T15:45:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f364fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.015+0000 7f364f400640 1 -- 172.21.15.90:0/3646490424 learned_addr learned my addr 172.21.15.90:0/3646490424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f3655200640 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1b1690 0x55683f1b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f364fe00640 1 -- 172.21.15.90:0/3646490424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1b1690 msgr2=0x55683f1b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f364fe00640 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1b1690 0x55683f1b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f364fe00640 1 -- 172.21.15.90:0/3646490424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 msgr2=0x55683f1b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f364fe00640 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 0x55683f1b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.016+0000 7f364fe00640 1 -- 172.21.15.90:0/3646490424 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683f0a48b0 con 0x55683f1abb30 2024-06-02T15:45:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.017+0000 7f364fe00640 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1abf50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3650105d20 tx=0x7f3638010db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be463a8e7046a90e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.017+0000 7f3645600640 1 -- 172.21.15.90:0/3646490424 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380e6060 con 0x55683f1abb30 2024-06-02T15:45:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.017+0000 7f3645600640 1 -- 172.21.15.90:0/3646490424 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36380a9260 con 0x55683f1abb30 2024-06-02T15:45:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.017+0000 7f3645600640 1 -- 172.21.15.90:0/3646490424 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380ad120 con 0x55683f1abb30 2024-06-02T15:45:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 -- 172.21.15.90:0/3646490424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 msgr2=0x55683f1abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1abf50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3650105d20 tx=0x7f3638010db0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 -- 172.21.15.90:0/3646490424 shutdown_connections 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1b1690 0x55683f1b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1ac520 0x55683f1b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.018+0000 7f365a01b740 1 --2- 172.21.15.90:0/3646490424 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1abf50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.019+0000 7f365a01b740 1 -- 172.21.15.90:0/3646490424 >> 172.21.15.90:0/3646490424 conn(0x55683f1a71a0 msgr2=0x55683f1a75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.019+0000 7f365a01b740 1 -- 172.21.15.90:0/3646490424 shutdown_connections 2024-06-02T15:45:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.019+0000 7f365a01b740 1 -- 172.21.15.90:0/3646490424 wait complete. 2024-06-02T15:45:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.050+0000 7f365a01b740 1 -- start start 2024-06-02T15:45:11.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.072+0000 7f365a01b740 1 Processor -- start 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.073+0000 7f365a01b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.073+0000 7f365a01b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.073+0000 7f365a01b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f365a01b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55683edf1790 con 0x55683f1b1690 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f365a01b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55683f0b3bc0 con 0x55683f1abb30 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f365a01b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55683f1b45e0 con 0x55683f1ac520 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37356/0 (socket says 172.21.15.90:37356) 2024-06-02T15:45:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364fe00640 1 -- 172.21.15.90:0/1327012125 learned_addr learned my addr 172.21.15.90:0/1327012125 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f3655200640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364f400640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364fe00640 1 -- 172.21.15.90:0/1327012125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 msgr2=0x55683f1b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.074+0000 7f364fe00640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f364fe00640 1 -- 172.21.15.90:0/1327012125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 msgr2=0x55683f1bce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f364fe00640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1bce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f364fe00640 1 -- 172.21.15.90:0/1327012125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683ee20210 con 0x55683f1abb30 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f364f400640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f364fe00640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36380a91f0 tx=0x7f363800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d710c5a9c135c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f363e000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f363e000640 1 -- 172.21.15.90:0/1327012125 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638010a90 con 0x55683f1abb30 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.075+0000 7f363e000640 1 -- 172.21.15.90:0/1327012125 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f36380dfb60 con 0x55683f1abb30 2024-06-02T15:45:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.076+0000 7f365a01b740 10 client.0 _ll_register_callbacks cb 0x55683f091400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.086+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36380a9260 con 0x55683f1abb30 2024-06-02T15:45:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.086+0000 7f363e000640 1 -- 172.21.15.90:0/1327012125 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f36380b20c0 con 0x55683f1abb30 2024-06-02T15:45:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.087+0000 7f365a01b740 10 client.25526 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.087+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55683eeec620 con 0x55683f1abb30 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.171+0000 7f363e000640 1 -- 172.21.15.90:0/1327012125 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f36380c3b50 con 0x55683f1abb30 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.171+0000 7f365a01b740 10 client.25526 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.171+0000 7f365a01b740 10 client.25526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.171+0000 7f365a01b740 20 client.25526 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.172+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f1c0e10 0x55683f1c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.172+0000 7f365a01b740 4 client.25526 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:45:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.172+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683f11dae0 con 0x55683f1c0e10 2024-06-02T15:45:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.172+0000 7f364f400640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f1c0e10 0x55683f1c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.173+0000 7f363e000640 10 client.25526 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.173+0000 7f364f400640 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f1c0e10 0x55683f1c3200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea87fe7c628d0832 server_cookie=257fe21347a9c98 in_seq=0 out_seq=0 2024-06-02T15:45:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.174+0000 7f363e000640 1 -- 172.21.15.90:0/1327012125 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55683f11dae0 con 0x55683f1c0e10 2024-06-02T15:45:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.174+0000 7f363e000640 10 client.25526 handle_command_reply: tid=0 2024-06-02T15:45:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.175+0000 7f365a01b740 1 client.25526 shutdown 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.175+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 msgr2=0x55683f1b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.175+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f36380a91f0 tx=0x7f363800cf90 comp rx=0 tx=0).stop 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.175+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 shutdown_connections 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.175+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f1c0e10 0x55683f1c3200 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1bce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f363e000640 0 client.25526 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 --2- 172.21.15.90:0/1327012125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1abb30 0x55683f1b58c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 >> 172.21.15.90:0/1327012125 conn(0x55683f1a71a0 msgr2=0x55683f1a6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 shutdown_connections 2024-06-02T15:45:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 1 -- 172.21.15.90:0/1327012125 wait complete. 2024-06-02T15:45:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.176+0000 7f365a01b740 20 client.25526 trim_cache size 0 max 16384 2024-06-02T15:45:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.223+0000 7f365a01b740 1 Processor -- start 2024-06-02T15:45:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.223+0000 7f365a01b740 1 -- start start 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1abb30 0x55683f1b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36501b0140 con 0x55683f1b1690 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36501ada10 con 0x55683f1ac520 2024-06-02T15:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f365a01b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36501ab860 con 0x55683f1abb30 2024-06-02T15:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f364f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f364f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37364/0 (socket says 172.21.15.90:37364) 2024-06-02T15:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f364f400640 1 -- 172.21.15.90:0/837434564 learned_addr learned my addr 172.21.15.90:0/837434564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.224+0000 7f364fe00640 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1abb30 0x55683f1b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 -- 172.21.15.90:0/837434564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1abb30 msgr2=0x55683f1b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1abb30 0x55683f1b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 -- 172.21.15.90:0/837434564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 msgr2=0x55683f1a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f3655200640 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1a5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 -- 172.21.15.90:0/837434564 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683f1a6fd0 con 0x55683f1ac520 2024-06-02T15:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f364f400640 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3640004650 tx=0x7f364003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5fd879957b6f69 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.225+0000 7f3644c00640 1 -- 172.21.15.90:0/837434564 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640035050 con 0x55683f1ac520 2024-06-02T15:45:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.226+0000 7f3644c00640 1 -- 172.21.15.90:0/837434564 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364005c070 con 0x55683f1ac520 2024-06-02T15:45:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.226+0000 7f3644c00640 1 -- 172.21.15.90:0/837434564 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364003c910 con 0x55683f1ac520 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 -- 172.21.15.90:0/837434564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 msgr2=0x55683f1bce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3640004650 tx=0x7f364003e000 comp rx=0 tx=0).stop 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 -- 172.21.15.90:0/837434564 shutdown_connections 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1b1690 0x55683f1a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1ac520 0x55683f1bce00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 --2- 172.21.15.90:0/837434564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1abb30 0x55683f1b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 -- 172.21.15.90:0/837434564 >> 172.21.15.90:0/837434564 conn(0x55683f1a71a0 msgr2=0x55683f1b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 -- 172.21.15.90:0/837434564 shutdown_connections 2024-06-02T15:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.227+0000 7f365a01b740 1 -- 172.21.15.90:0/837434564 wait complete. 2024-06-02T15:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.228+0000 7f365a01b740 1 -- start start 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.228+0000 7f365a01b740 1 Processor -- start 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1abb30 0x55683f1ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1b1690 0x55683f1bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36501abed0 con 0x55683f1abb30 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36501ad340 con 0x55683f1b1690 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.229+0000 7f365a01b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36501ac5a0 con 0x55683f1ac520 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f365a01b740 10 client.0 _ll_register_callbacks cb 0x7f36480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1abb30 0x55683f1ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:59114/0 (socket says 172.21.15.90:59114) 2024-06-02T15:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 -- 172.21.15.90:0/2179363568 learned_addr learned my addr 172.21.15.90:0/2179363568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f3655200640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1b1690 0x55683f1bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 -- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1b1690 msgr2=0x55683f1bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1b1690 0x55683f1bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 -- 172.21.15.90:0/2179363568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1abb30 msgr2=0x55683f1ba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1abb30 0x55683f1ba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.230+0000 7f364f400640 1 -- 172.21.15.90:0/2179363568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683f1b4980 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f364f400640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3640003660 tx=0x7f364002ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8397baabae640a99 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f363e000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640065070 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3640005440 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364005c070 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3640035040 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f365a01b740 10 client.25495 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.231+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55683f1bccd0 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f364005c070 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f3640031930 con 0x55683f1ac520 2024-06-02T15:45:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 10 client.25495 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 10 client.25495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 20 client.25495 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f22bbe0 0x55683f22e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 4 client.25495 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.232+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55683f1172c0 con 0x55683f22bbe0 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.233+0000 7f364fe00640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f22bbe0 0x55683f22e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.233+0000 7f364fe00640 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f22bbe0 0x55683f22e000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd1e750880c759a server_cookie=d70cc7ef3404c6cf in_seq=0 out_seq=0 2024-06-02T15:45:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.233+0000 7f363e000640 10 client.25495 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.252+0000 7f363e000640 1 -- 172.21.15.90:0/2179363568 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55683f1172c0 con 0x55683f22bbe0 2024-06-02T15:45:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.252+0000 7f363e000640 10 client.25495 handle_command_reply: tid=0 2024-06-02T15:45:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.252+0000 7f365a01b740 1 client.25495 shutdown 2024-06-02T15:45:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 msgr2=0x55683f1ba620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3640003660 tx=0x7f364002ab80 comp rx=0 tx=0).stop 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 shutdown_connections 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55683f22bbe0 0x55683f22e000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55683f1b1690 0x55683f1bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55683f1ac520 0x55683f1ba620 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f363e000640 0 client.25495 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 --2- 172.21.15.90:0/2179363568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55683f1abb30 0x55683f1ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 >> 172.21.15.90:0/2179363568 conn(0x55683f1a71a0 msgr2=0x55683f1acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 shutdown_connections 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 1 -- 172.21.15.90:0/2179363568 wait complete. 2024-06-02T15:45:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.253+0000 7f365a01b740 20 client.25495 trim_cache size 0 max 16384 2024-06-02T15:45:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:45:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:45:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:45:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 msgr2=0x7f36340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 0x7f36340b4eb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f365019c380 tx=0x7f3640005a20 comp rx=0 tx=0).stop 2024-06-02T15:45:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 msgr2=0x7f365019ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 0x7f365019ae70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f363800e9c0 tx=0x7f363800ee80 comp rx=0 tx=0).stop 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 shutdown_connections 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f36340b29f0 0x7f36340b4eb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3650106880 0x7f36501a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3650105ec0 0x7f365019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 --2- 172.21.15.90:0/3524662587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3650104cd0 0x7f365019ae70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.259+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 >> 172.21.15.90:0/3524662587 conn(0x7f3650100240 msgr2=0x7f36501091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.260+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 shutdown_connections 2024-06-02T15:45:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:11.260+0000 7f3657200640 1 -- 172.21.15.90:0/3524662587 wait complete. 2024-06-02T15:45:11.269 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:11.269 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:11.301 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:11.301 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:11.301 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 6 -lt 100 ']' 2024-06-02T15:45:11.303 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 6 + 1 2024-06-02T15:45:11.377 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=7 2024-06-02T15:45:11.377 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:11.447 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:11.448 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:11.452 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:11.453 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:11.454 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:11.619 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:11 smithi135 ceph-mon[36839]: pgmap v536: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-06-02T15:45:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:11 smithi090 ceph-mon[31595]: pgmap v536: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-06-02T15:45:11.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:11 smithi120 ceph-mon[36451]: pgmap v536: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 KiB/s wr, 8 op/s 2024-06-02T15:45:12.139 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:45:12.140 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:45:12.340 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:12.340 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:12.571 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:12.571 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:12.605 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:12.605 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:12.605 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 7 -lt 100 ']' 2024-06-02T15:45:12.606 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 7 + 1 2024-06-02T15:45:12.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=8 2024-06-02T15:45:12.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:12.697 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:12.698 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:12.701 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:12.703 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:12.703 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:13.540 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:13 smithi090 ceph-mon[31595]: pgmap v537: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-02T15:45:13.740 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:13.741 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:13 smithi120 ceph-mon[36451]: pgmap v537: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-02T15:45:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:13 smithi135 ceph-mon[36839]: pgmap v537: 129 pgs: 129 active+clean; 741 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.7 KiB/s wr, 7 op/s 2024-06-02T15:45:14.051 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:14.051 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:14.069 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:14.069 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:14.069 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 8 -lt 100 ']' 2024-06-02T15:45:14.070 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 8 + 1 2024-06-02T15:45:14.106 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=9 2024-06-02T15:45:14.106 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:14.215 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:14.216 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:14.220 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:14.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:14.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:15.332 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:15.332 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:15 smithi090 ceph-mon[31595]: pgmap v538: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-06-02T15:45:15.624 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:15.624 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:15.648 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:15.648 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:15.648 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 9 -lt 100 ']' 2024-06-02T15:45:15.649 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 9 + 1 2024-06-02T15:45:15.681 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=10 2024-06-02T15:45:15.681 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:15.747 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:15.748 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:15.752 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:15.753 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:15.754 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:15.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:15 smithi120 ceph-mon[36451]: pgmap v538: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-06-02T15:45:15.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:15 smithi135 ceph-mon[36839]: pgmap v538: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 9 op/s 2024-06-02T15:45:16.998 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:16.998 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:17.448 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:17.449 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:17.482 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:17.482 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 10 -lt 100 ']' 2024-06-02T15:45:17.483 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 10 + 1 2024-06-02T15:45:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:17 smithi090 ceph-mon[31595]: pgmap v539: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:45:17.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=11 2024-06-02T15:45:17.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:17.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:17 smithi120 ceph-mon[36451]: pgmap v539: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:45:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:17 smithi135 ceph-mon[36839]: pgmap v539: 129 pgs: 129 active+clean; 780 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:45:17.922 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:17.923 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:17.927 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:17.928 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:17.929 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:19.374 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:19.375 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:19 smithi090 ceph-mon[31595]: pgmap v540: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:19 smithi120 ceph-mon[36451]: pgmap v540: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:19 smithi135 ceph-mon[36839]: pgmap v540: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:19.865 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:19.865 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:19.893 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:19.893 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:19.893 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 11 -lt 100 ']' 2024-06-02T15:45:19.895 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 11 + 1 2024-06-02T15:45:19.964 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=12 2024-06-02T15:45:19.964 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:20.106 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:20.106 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:20.111 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:20.112 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:20.113 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:21.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:21 smithi090 ceph-mon[31595]: pgmap v541: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:45:21.742 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:21.742 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:21 smithi120 ceph-mon[36451]: pgmap v541: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:45:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:21 smithi135 ceph-mon[36839]: pgmap v541: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:45:22.290 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:22.290 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:22.340 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:22.340 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:22.340 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 12 -lt 100 ']' 2024-06-02T15:45:22.342 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 12 + 1 2024-06-02T15:45:22.443 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=13 2024-06-02T15:45:22.443 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:22.511 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:22.512 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:22.516 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:22.517 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:22.518 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:23 smithi090 ceph-mon[31595]: pgmap v542: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-02T15:45:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:23 smithi120 ceph-mon[36451]: pgmap v542: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-02T15:45:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:23 smithi135 ceph-mon[36839]: pgmap v542: 129 pgs: 129 active+clean; 832 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-06-02T15:45:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:25 smithi090 ceph-mon[31595]: pgmap v543: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:45:25.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:25 smithi120 ceph-mon[36451]: pgmap v543: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:45:25.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:25 smithi135 ceph-mon[36839]: pgmap v543: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:45:26.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:26 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:45:26.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:26 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:45:26.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:26 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:45:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:27 smithi090 ceph-mon[31595]: pgmap v544: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:45:27.648 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:27.648 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:27.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:27 smithi120 ceph-mon[36451]: pgmap v544: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:45:27.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:27 smithi135 ceph-mon[36839]: pgmap v544: 129 pgs: 129 active+clean; 854 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:45:28.102 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:28.103 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:28.236 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:28.236 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:28.236 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 13 -lt 100 ']' 2024-06-02T15:45:28.238 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 13 + 1 2024-06-02T15:45:28.493 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=14 2024-06-02T15:45:28.493 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:28.545 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:28.545 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:28.550 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:28.550 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:28.551 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:29.652 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:29 smithi090 ceph-mon[31595]: pgmap v545: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:45:29.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:29 smithi120 ceph-mon[36451]: pgmap v545: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:45:29.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:29 smithi135 ceph-mon[36839]: pgmap v545: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:45:30.198 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:30.198 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:30.627 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:30.627 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:30.675 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:30.675 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:30.675 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 14 -lt 100 ']' 2024-06-02T15:45:30.676 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 14 + 1 2024-06-02T15:45:30.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:30 smithi120 ceph-mon[36451]: pgmap v546: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-02T15:45:30.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:30 smithi135 ceph-mon[36839]: pgmap v546: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-02T15:45:30.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:30 smithi090 ceph-mon[31595]: pgmap v546: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-06-02T15:45:30.963 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=15 2024-06-02T15:45:30.963 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:31.229 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:31.229 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:31.234 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:31.235 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:31.236 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 -- 172.21.15.90:0/1710888291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc378100700 msgr2=0x7fc378100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 --2- 172.21.15.90:0/1710888291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc378100700 0x7fc378100b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc368009900 tx=0x7fc36802a400 comp rx=0 tx=0).stop 2024-06-02T15:45:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 -- 172.21.15.90:0/1710888291 shutdown_connections 2024-06-02T15:45:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 --2- 172.21.15.90:0/1710888291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc37810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 --2- 172.21.15.90:0/1710888291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3781010f0 0x7fc378108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 --2- 172.21.15.90:0/1710888291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc378100700 0x7fc378100b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 -- 172.21.15.90:0/1710888291 >> 172.21.15.90:0/1710888291 conn(0x7fc3780fc270 msgr2=0x7fc3780fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.423+0000 7fc37d600640 1 -- 172.21.15.90:0/1710888291 shutdown_connections 2024-06-02T15:45:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.424+0000 7fc37d600640 1 -- 172.21.15.90:0/1710888291 wait complete. 2024-06-02T15:45:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.596+0000 7fc37d600640 1 Processor -- start 2024-06-02T15:45:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.597+0000 7fc37d600640 1 -- start start 2024-06-02T15:45:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 0x7fc37819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3781010f0 0x7fc37819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810e640 con 0x7fc378100700 2024-06-02T15:45:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810e4c0 con 0x7fc3781010f0 2024-06-02T15:45:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.598+0000 7fc37d600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc37810e7c0 con 0x7fc378108780 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc376000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3781010f0 0x7fc37819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc376a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 0x7fc37819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52920/0 (socket says 172.21.15.90:52920) 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 -- 172.21.15.90:0/1824403587 learned_addr learned my addr 172.21.15.90:0/1824403587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 -- 172.21.15.90:0/1824403587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3781010f0 msgr2=0x7fc37819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3781010f0 0x7fc37819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 -- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 msgr2=0x7fc37819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.599+0000 7fc377400640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 0x7fc37819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.600+0000 7fc377400640 1 -- 172.21.15.90:0/1824403587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3781a2a50 con 0x7fc378108780 2024-06-02T15:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.600+0000 7fc376a00640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 0x7fc37819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.600+0000 7fc377400640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc37000eb40 tx=0x7fc37000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.601+0000 7fc36f400640 1 -- 172.21.15.90:0/1824403587 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370004b20 con 0x7fc378108780 2024-06-02T15:45:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.601+0000 7fc36f400640 1 -- 172.21.15.90:0/1824403587 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc370004cc0 con 0x7fc378108780 2024-06-02T15:45:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.601+0000 7fc36f400640 1 -- 172.21.15.90:0/1824403587 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370010720 con 0x7fc378108780 2024-06-02T15:45:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.601+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3781a2d40 con 0x7fc378108780 2024-06-02T15:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.609+0000 7fc36f400640 1 -- 172.21.15.90:0/1824403587 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc370005560 con 0x7fc378108780 2024-06-02T15:45:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.612+0000 7fc36f400640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 0x7fc35c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.612+0000 7fc376a00640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 0x7fc35c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.613+0000 7fc376a00640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 0x7fc35c0b4de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc368031500 tx=0x7fc368005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.619+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3781a3280 con 0x7fc378108780 2024-06-02T15:45:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.619+0000 7fc36f400640 1 -- 172.21.15.90:0/1824403587 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc3700a5850 con 0x7fc378108780 2024-06-02T15:45:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.885+0000 7fc38047b740 1 Processor -- start 2024-06-02T15:45:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.885+0000 7fc38047b740 1 -- start start 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba821f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 0x55c7ba829b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba791b90 con 0x55c7ba821b30 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba781730 con 0x55c7ba822520 2024-06-02T15:45:32.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc38047b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba785770 con 0x55c7ba827690 2024-06-02T15:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc376000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc377400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 0x55c7ba829b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc376000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39740/0 (socket says 172.21.15.90:39740) 2024-06-02T15:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.886+0000 7fc376000640 1 -- 172.21.15.90:0/1653865431 learned_addr learned my addr 172.21.15.90:0/1653865431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376a00640 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba821f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376000640 1 -- 172.21.15.90:0/1653865431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 msgr2=0x55c7ba829b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376000640 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 0x55c7ba829b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376000640 1 -- 172.21.15.90:0/1653865431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 msgr2=0x55c7ba821f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376000640 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba821f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc376000640 1 -- 172.21.15.90:0/1653865431 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ba71a8b0 con 0x55c7ba822520 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.887+0000 7fc377400640 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 0x55c7ba829b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.888+0000 7fc376000640 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc364009c90 tx=0x7fc364011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8855051bea33257b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.888+0000 7fc363400640 1 -- 172.21.15.90:0/1653865431 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36401e070 con 0x55c7ba822520 2024-06-02T15:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.888+0000 7fc363400640 1 -- 172.21.15.90:0/1653865431 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc364019ae0 con 0x55c7ba822520 2024-06-02T15:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.888+0000 7fc363400640 1 -- 172.21.15.90:0/1653865431 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36400e9c0 con 0x55c7ba822520 2024-06-02T15:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 -- 172.21.15.90:0/1653865431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 msgr2=0x55c7ba827020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc364009c90 tx=0x7fc364011810 comp rx=0 tx=0).stop 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 -- 172.21.15.90:0/1653865431 shutdown_connections 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba827690 0x55c7ba829b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba822520 0x55c7ba827020 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 --2- 172.21.15.90:0/1653865431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba821f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 -- 172.21.15.90:0/1653865431 >> 172.21.15.90:0/1653865431 conn(0x55c7ba81d1a0 msgr2=0x55c7ba81d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.889+0000 7fc38047b740 1 -- 172.21.15.90:0/1653865431 shutdown_connections 2024-06-02T15:45:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.890+0000 7fc38047b740 1 -- 172.21.15.90:0/1653865431 wait complete. 2024-06-02T15:45:32.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.918+0000 7fc38047b740 1 -- start start 2024-06-02T15:45:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.946+0000 7fc38047b740 1 Processor -- start 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.947+0000 7fc38047b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba82b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.947+0000 7fc38047b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc38047b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba832e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc38047b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba467790 con 0x55c7ba821b30 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc38047b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba729bc0 con 0x55c7ba827690 2024-06-02T15:45:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc38047b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba82a640 con 0x55c7ba822520 2024-06-02T15:45:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc376000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc377400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba832e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc376000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52928/0 (socket says 172.21.15.90:52928) 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc376000640 1 -- 172.21.15.90:0/744686887 learned_addr learned my addr 172.21.15.90:0/744686887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.948+0000 7fc376a00640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba82b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 -- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 msgr2=0x55c7ba832e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba832e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 -- 172.21.15.90:0/744686887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 msgr2=0x55c7ba82b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba82b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 -- 172.21.15.90:0/744686887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ba496210 con 0x55c7ba822520 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc377400640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba832e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc376000640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc364019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975fb9b83db77fa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.949+0000 7fc357e00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.950+0000 7fc357e00640 1 -- 172.21.15.90:0/744686887 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36401e040 con 0x55c7ba822520 2024-06-02T15:45:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.950+0000 7fc357e00640 1 -- 172.21.15.90:0/744686887 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc364004030 con 0x55c7ba822520 2024-06-02T15:45:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.954+0000 7fc38047b740 10 client.0 _ll_register_callbacks cb 0x55c7ba707400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.955+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc364019ae0 con 0x55c7ba822520 2024-06-02T15:45:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.984+0000 7fc357e00640 1 -- 172.21.15.90:0/744686887 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc364033050 con 0x55c7ba822520 2024-06-02T15:45:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.985+0000 7fc38047b740 10 client.25507 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:32.985+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55c7ba562620 con 0x55c7ba822520 2024-06-02T15:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc357e00640 1 -- 172.21.15.90:0/744686887 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc364004230 con 0x55c7ba822520 2024-06-02T15:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 10 client.25507 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 10 client.25507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 20 client.25507 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba836e90 0x55c7ba839320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 4 client.25507 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.017+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ba793ae0 con 0x55c7ba836e90 2024-06-02T15:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.018+0000 7fc376a00640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba836e90 0x55c7ba839320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.018+0000 7fc376a00640 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba836e90 0x55c7ba839320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fb4902fb037358 server_cookie=a27370df8823b2ee in_seq=0 out_seq=0 2024-06-02T15:45:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.018+0000 7fc357e00640 10 client.25507 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc357e00640 1 -- 172.21.15.90:0/744686887 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c7ba793ae0 con 0x55c7ba836e90 2024-06-02T15:45:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc357e00640 10 client.25507 handle_command_reply: tid=0 2024-06-02T15:45:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc38047b740 1 client.25507 shutdown 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 msgr2=0x55c7ba82be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc364000c00 tx=0x7fc364019660 comp rx=0 tx=0).stop 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 shutdown_connections 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.020+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba836e90 0x55c7ba839320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba832e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc357e00640 0 client.25507 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba822520 0x55c7ba82be30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 --2- 172.21.15.90:0/744686887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba821b30 0x55c7ba82b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 >> 172.21.15.90:0/744686887 conn(0x55c7ba81d1a0 msgr2=0x55c7ba81c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 shutdown_connections 2024-06-02T15:45:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 1 -- 172.21.15.90:0/744686887 wait complete. 2024-06-02T15:45:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.021+0000 7fc38047b740 20 client.25507 trim_cache size 0 max 16384 2024-06-02T15:45:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.047+0000 7fc38047b740 1 Processor -- start 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 -- start start 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba832e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba820190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781b0110 con 0x55c7ba822520 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781ad9e0 con 0x55c7ba827690 2024-06-02T15:45:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.048+0000 7fc38047b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781ab830 con 0x55c7ba821b30 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba832e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52940/0 (socket says 172.21.15.90:52940) 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 -- 172.21.15.90:0/1352910582 learned_addr learned my addr 172.21.15.90:0/1352910582 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc377400640 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba820190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 -- 172.21.15.90:0/1352910582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 msgr2=0x55c7ba820190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba820190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 -- 172.21.15.90:0/1352910582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 msgr2=0x55c7ba832e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba832e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc376a00640 1 -- 172.21.15.90:0/1352910582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ba81cfd0 con 0x55c7ba821b30 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.049+0000 7fc377400640 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba820190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.050+0000 7fc376a00640 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc368009230 tx=0x7fc36803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac97d89397cb7e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.050+0000 7fc362a00640 1 -- 172.21.15.90:0/1352910582 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368035050 con 0x55c7ba821b30 2024-06-02T15:45:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.050+0000 7fc362a00640 1 -- 172.21.15.90:0/1352910582 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc368059070 con 0x55c7ba821b30 2024-06-02T15:45:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.050+0000 7fc362a00640 1 -- 172.21.15.90:0/1352910582 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36804b440 con 0x55c7ba821b30 2024-06-02T15:45:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.051+0000 7fc38047b740 1 -- 172.21.15.90:0/1352910582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 msgr2=0x55c7ba829a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.051+0000 7fc38047b740 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc368009230 tx=0x7fc36803e000 comp rx=0 tx=0).stop 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 -- 172.21.15.90:0/1352910582 shutdown_connections 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba820190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba832e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 --2- 172.21.15.90:0/1352910582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba829a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 -- 172.21.15.90:0/1352910582 >> 172.21.15.90:0/1352910582 conn(0x55c7ba81d1a0 msgr2=0x55c7ba817dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 -- 172.21.15.90:0/1352910582 shutdown_connections 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 -- 172.21.15.90:0/1352910582 wait complete. 2024-06-02T15:45:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.052+0000 7fc38047b740 1 -- start start 2024-06-02T15:45:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 Processor -- start 2024-06-02T15:45:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba830180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba89faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781abea0 con 0x55c7ba822520 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781ad310 con 0x55c7ba827690 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3781ac570 con 0x55c7ba821b30 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc38047b740 10 client.0 _ll_register_callbacks cb 0x7fc36401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc376a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc376a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52946/0 (socket says 172.21.15.90:52946) 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.053+0000 7fc376a00640 1 -- 172.21.15.90:0/3558652502 learned_addr learned my addr 172.21.15.90:0/3558652502 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc377400640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba89faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376000640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba830180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 -- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 msgr2=0x55c7ba89faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba89faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 -- 172.21.15.90:0/3558652502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 msgr2=0x55c7ba830180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba830180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 -- 172.21.15.90:0/3558652502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ba82c1a0 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc377400640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba89faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376000640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba830180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc376a00640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc368005ab0 tx=0x7fc368009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b70a591534b176c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.054+0000 7fc357e00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368061070 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc368059070 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc3680545a0 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc36806f050 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc38047b740 10 client.25513 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7ba8a1000 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.055+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc368059070 con 0x55c7ba821b30 2024-06-02T15:45:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc368066870 con 0x55c7ba821b30 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 10 client.25513 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 20 client.25513 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba8a1900 0x55c7ba8a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 4 client.25513 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7ba78d2c0 con 0x55c7ba8a1900 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.056+0000 7fc376000640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba8a1900 0x55c7ba8a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.057+0000 7fc357e00640 10 client.25513 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.057+0000 7fc376000640 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba8a1900 0x55c7ba8a3d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f089db3816575 server_cookie=113bbece8de9a9e in_seq=0 out_seq=0 2024-06-02T15:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.077+0000 7fc357e00640 1 -- 172.21.15.90:0/3558652502 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c7ba78d2c0 con 0x55c7ba8a1900 2024-06-02T15:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.077+0000 7fc357e00640 10 client.25513 handle_command_reply: tid=0 2024-06-02T15:45:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.077+0000 7fc38047b740 1 client.25513 shutdown 2024-06-02T15:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 msgr2=0x55c7ba82fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc368005ab0 tx=0x7fc368009630 comp rx=0 tx=0).stop 2024-06-02T15:45:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 shutdown_connections 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c7ba8a1900 0x55c7ba8a3d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c7ba827690 0x55c7ba89faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c7ba822520 0x55c7ba830180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc357e00640 0 client.25513 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 --2- 172.21.15.90:0/3558652502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ba821b30 0x55c7ba82fc40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 >> 172.21.15.90:0/3558652502 conn(0x55c7ba81d1a0 msgr2=0x55c7ba820190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 shutdown_connections 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 1 -- 172.21.15.90:0/3558652502 wait complete. 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.078+0000 7fc38047b740 20 client.25513 trim_cache size 0 max 16384 2024-06-02T15:45:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 msgr2=0x7fc35c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 0x7fc35c0b4de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc368031500 tx=0x7fc368005a20 comp rx=0 tx=0).stop 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 msgr2=0x7fc3781a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc37000eb40 tx=0x7fc37000c2d0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 shutdown_connections 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc35c0b2920 0x7fc35c0b4de0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc378108780 0x7fc3781a23e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc3781010f0 0x7fc37819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.079+0000 7fc37d600640 1 --2- 172.21.15.90:0/1824403587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc378100700 0x7fc37819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.080+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 >> 172.21.15.90:0/1824403587 conn(0x7fc3780fc270 msgr2=0x7fc3780fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.080+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 shutdown_connections 2024-06-02T15:45:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:33.080+0000 7fc37d600640 1 -- 172.21.15.90:0/1824403587 wait complete. 2024-06-02T15:45:33.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:33 smithi120 ceph-mon[36451]: pgmap v547: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-06-02T15:45:33.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:33 smithi135 ceph-mon[36839]: pgmap v547: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-06-02T15:45:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:33 smithi090 ceph-mon[31595]: pgmap v547: 129 pgs: 129 active+clean; 881 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.0 KiB/s wr, 7 op/s 2024-06-02T15:45:33.611 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:33.611 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:34.323 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:34.323 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:34.409 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:34.409 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:34.409 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 15 -lt 100 ']' 2024-06-02T15:45:34.410 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 15 + 1 2024-06-02T15:45:34.616 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=16 2024-06-02T15:45:34.617 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:34.722 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:34.722 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:34.727 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:34.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:34.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:35.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:35 smithi120 ceph-mon[36451]: pgmap v548: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-06-02T15:45:35.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:35 smithi135 ceph-mon[36839]: pgmap v548: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-06-02T15:45:35.344 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:45:35.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:35 smithi090 ceph-mon[31595]: pgmap v548: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 KiB/s wr, 8 op/s 2024-06-02T15:45:36.431 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:36.431 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:36.795 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:36.795 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:36.851 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:36.851 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:36.851 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 16 -lt 100 ']' 2024-06-02T15:45:36.853 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 16 + 1 2024-06-02T15:45:37.092 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=17 2024-06-02T15:45:37.092 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:37.152 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:37.152 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:37.156 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:37.157 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:37.158 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:37 smithi120 ceph-mon[36451]: pgmap v549: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s 2024-06-02T15:45:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:45:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:45:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:45:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:37 smithi135 ceph-mon[36839]: pgmap v549: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s 2024-06-02T15:45:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:45:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:45:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:45:37.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:37 smithi090 ceph-mon[31595]: pgmap v549: 129 pgs: 129 active+clean; 906 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.4 KiB/s wr, 12 op/s 2024-06-02T15:45:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:45:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:45:37.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:45:39.206 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:39.206 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:39.264 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:39 smithi090 ceph-mon[31595]: pgmap v550: 129 pgs: 129 active+clean; 953 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:39 smithi120 ceph-mon[36451]: pgmap v550: 129 pgs: 129 active+clean; 953 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:39 smithi135 ceph-mon[36839]: pgmap v550: 129 pgs: 129 active+clean; 953 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:39.611 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:39.612 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:39.659 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:39.659 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:39.659 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 17 -lt 100 ']' 2024-06-02T15:45:39.660 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 17 + 1 2024-06-02T15:45:39.905 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=18 2024-06-02T15:45:39.906 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:40.156 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:40.156 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:40.161 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:40.162 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:40.163 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:41 smithi120 ceph-mon[36451]: pgmap v551: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-02T15:45:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:41 smithi135 ceph-mon[36839]: pgmap v551: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-02T15:45:41.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:41 smithi090 ceph-mon[31595]: pgmap v551: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-06-02T15:45:41.930 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:41.931 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:42.406 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:42.406 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:42.436 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:42.436 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:42.437 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 18 -lt 100 ']' 2024-06-02T15:45:42.438 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 18 + 1 2024-06-02T15:45:42.534 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=19 2024-06-02T15:45:42.534 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:42.610 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:42.611 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:42.616 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:42.617 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:42.617 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:43.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:43 smithi120 ceph-mon[36451]: pgmap v552: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-02T15:45:43.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:43 smithi135 ceph-mon[36839]: pgmap v552: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-02T15:45:43.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:43 smithi090 ceph-mon[31595]: pgmap v552: 129 pgs: 129 active+clean; 954 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-06-02T15:45:43.997 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:43.998 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:44.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:44.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:44.572 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:44.572 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:44.572 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 19 -lt 100 ']' 2024-06-02T15:45:44.573 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 19 + 1 2024-06-02T15:45:44.683 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=20 2024-06-02T15:45:44.684 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:44.762 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:44.763 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:44.767 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:44.768 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:44.769 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:45.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:45 smithi120 ceph-mon[36451]: pgmap v553: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:45.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:45 smithi135 ceph-mon[36839]: pgmap v553: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:45.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:45 smithi090 ceph-mon[31595]: pgmap v553: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:45:46.180 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:46.181 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:46.592 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:46.593 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:46.609 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:46.609 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:46.609 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 20 -lt 100 ']' 2024-06-02T15:45:46.610 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 20 + 1 2024-06-02T15:45:46.750 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=21 2024-06-02T15:45:46.750 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:46.905 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:46.905 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:46.911 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:46.911 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:46.912 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:47.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:47 smithi135 ceph-mon[36839]: pgmap v554: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-06-02T15:45:47.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:47 smithi090 ceph-mon[31595]: pgmap v554: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-06-02T15:45:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:47 smithi120 ceph-mon[36451]: pgmap v554: 129 pgs: 129 active+clean; 986 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-06-02T15:45:48.280 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:48.280 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:48.698 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:48.698 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:48.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:48.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:48.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 21 -lt 100 ']' 2024-06-02T15:45:48.729 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 21 + 1 2024-06-02T15:45:48.983 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=22 2024-06-02T15:45:48.983 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:49.188 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:49.188 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:49.194 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:49.195 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:49.195 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:49.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:49 smithi135 ceph-mon[36839]: pgmap v555: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:45:49.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:49 smithi090 ceph-mon[31595]: pgmap v555: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:45:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:49 smithi120 ceph-mon[36451]: pgmap v555: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:45:51.080 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:51.080 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:51.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:51 smithi090 ceph-mon[31595]: pgmap v556: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:45:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:51 smithi120 ceph-mon[36451]: pgmap v556: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:45:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:51 smithi135 ceph-mon[36839]: pgmap v556: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:45:52.031 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:52.031 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:52.153 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:52.154 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:52.154 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 22 -lt 100 ']' 2024-06-02T15:45:52.155 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 22 + 1 2024-06-02T15:45:52.255 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=23 2024-06-02T15:45:52.255 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:52.337 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:52.337 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:52.341 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:52.342 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:52.343 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:53.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:53 smithi090 ceph-mon[31595]: pgmap v557: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:45:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:53 smithi120 ceph-mon[36451]: pgmap v557: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:45:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:53 smithi135 ceph-mon[36839]: pgmap v557: 129 pgs: 129 active+clean; 1019 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:45:53.838 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:53.839 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:54.380 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:54.380 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:54.509 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:54.509 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:54.509 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 23 -lt 100 ']' 2024-06-02T15:45:54.511 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 23 + 1 2024-06-02T15:45:54.581 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=24 2024-06-02T15:45:54.581 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:54.651 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:54.651 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:54.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:54.656 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:54.656 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 -- 172.21.15.90:0/193206010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f00754b0 msgr2=0x7f71f0075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 --2- 172.21.15.90:0/193206010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f00754b0 0x7f71f0075930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009900 tx=0x7f71d802a400 comp rx=0 tx=0).stop 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 -- 172.21.15.90:0/193206010 shutdown_connections 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 --2- 172.21.15.90:0/193206010 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f010f080 0x7f71f0111470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 --2- 172.21.15.90:0/193206010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f00754b0 0x7f71f0075930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 --2- 172.21.15.90:0/193206010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.130+0000 7f71f5a00640 1 -- 172.21.15.90:0/193206010 >> 172.21.15.90:0/193206010 conn(0x7f71f00fe040 msgr2=0x7f71f0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.131+0000 7f71f5a00640 1 -- 172.21.15.90:0/193206010 shutdown_connections 2024-06-02T15:45:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.131+0000 7f71f5a00640 1 -- 172.21.15.90:0/193206010 wait complete. 2024-06-02T15:45:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.212+0000 7f71f5a00640 1 Processor -- start 2024-06-02T15:45:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.212+0000 7f71f5a00640 1 -- start start 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.213+0000 7f71f5a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f00754b0 0x7f71f019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.213+0000 7f71f5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.213+0000 7f71f5a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 0x7f71f01a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71f5a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0114c40 con 0x7f71f0076a90 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71f5a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0114ac0 con 0x7f71f00754b0 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71f5a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0114dc0 con 0x7f71f010f080 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71e6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71e6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58328/0 (socket says 172.21.15.90:58328) 2024-06-02T15:45:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71e6000640 1 -- 172.21.15.90:0/1149530297 learned_addr learned my addr 172.21.15.90:0/1149530297 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71eea00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f00754b0 0x7f71f019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.214+0000 7f71ef400640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 0x7f71f01a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 -- 172.21.15.90:0/1149530297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 msgr2=0x7f71f01a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 0x7f71f01a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 -- 172.21.15.90:0/1149530297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f00754b0 msgr2=0x7f71f019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f00754b0 0x7f71f019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 -- 172.21.15.90:0/1149530297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f01a6e30 con 0x7f71f0076a90 2024-06-02T15:45:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71ef400640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 0x7f71f01a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:45:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.215+0000 7f71e6000640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71d8031490 tx=0x7f71d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.224+0000 7f71e7e00640 1 -- 172.21.15.90:0/1149530297 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d803f070 con 0x7f71f0076a90 2024-06-02T15:45:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.225+0000 7f71e7e00640 1 -- 172.21.15.90:0/1149530297 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d8031bf0 con 0x7f71f0076a90 2024-06-02T15:45:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.225+0000 7f71e7e00640 1 -- 172.21.15.90:0/1149530297 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8035030 con 0x7f71f0076a90 2024-06-02T15:45:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.225+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f01a70c0 con 0x7f71f0076a90 2024-06-02T15:45:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.230+0000 7f71e7e00640 1 -- 172.21.15.90:0/1149530297 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f71d8042070 con 0x7f71f0076a90 2024-06-02T15:45:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.232+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f01a7560 con 0x7f71f0076a90 2024-06-02T15:45:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.233+0000 7f71e7e00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 0x7f71d40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.233+0000 7f71e7e00640 1 -- 172.21.15.90:0/1149530297 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71d8106300 con 0x7f71f0076a90 2024-06-02T15:45:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.233+0000 7f71eea00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 0x7f71d40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.234+0000 7f71eea00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 0x7f71d40b4eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71e00095f0 tx=0x7f71e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.359 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:55 smithi120 ceph-mon[36451]: pgmap v558: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-02T15:45:55.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:55 smithi090 ceph-mon[31595]: pgmap v558: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-02T15:45:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 Processor -- start 2024-06-02T15:45:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 -- start start 2024-06-02T15:45:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f966b30 0x55863f966f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55863f8d6b90 con 0x55863f967520 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55863f8c6730 con 0x55863f966b30 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.490+0000 7f71f87eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55863f8ca770 con 0x55863f96c690 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71e6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33706/0 (socket says 172.21.15.90:33706) 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 -- 172.21.15.90:0/3061037231 learned_addr learned my addr 172.21.15.90:0/3061037231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71e6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58338/0 (socket says 172.21.15.90:58338) 2024-06-02T15:45:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71eea00640 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f966b30 0x55863f966f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 -- 172.21.15.90:0/3061037231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f966b30 msgr2=0x55863f966f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f966b30 0x55863f966f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 -- 172.21.15.90:0/3061037231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 msgr2=0x55863f96c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.491+0000 7f71ef400640 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71ef400640 1 -- 172.21.15.90:0/3061037231 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863f85f8b0 con 0x55863f96c690 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71e6000640 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71ef400640 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fd0 tx=0x7f71e80108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3e8f446b67d5c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71dfe00640 1 -- 172.21.15.90:0/3061037231 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8011760 con 0x55863f96c690 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71dfe00640 1 -- 172.21.15.90:0/3061037231 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e8011db0 con 0x55863f96c690 2024-06-02T15:45:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.492+0000 7f71dfe00640 1 -- 172.21.15.90:0/3061037231 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800e3c0 con 0x55863f96c690 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 -- 172.21.15.90:0/3061037231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 msgr2=0x55863f96eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fd0 tx=0x7f71e80108e0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 -- 172.21.15.90:0/3061037231 shutdown_connections 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f96eb20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f96c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 --2- 172.21.15.90:0/3061037231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f966b30 0x55863f966f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.493+0000 7f71f87eb740 1 -- 172.21.15.90:0/3061037231 >> 172.21.15.90:0/3061037231 conn(0x55863f9621a0 msgr2=0x55863f9625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.494+0000 7f71f87eb740 1 -- 172.21.15.90:0/3061037231 shutdown_connections 2024-06-02T15:45:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.494+0000 7f71f87eb740 1 -- 172.21.15.90:0/3061037231 wait complete. 2024-06-02T15:45:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.536+0000 7f71f87eb740 1 -- start start 2024-06-02T15:45:55.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.553+0000 7f71f87eb740 1 Processor -- start 2024-06-02T15:45:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f970e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 0x55863f977e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55863f5ac790 con 0x55863f96c690 2024-06-02T15:45:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55863f86ebc0 con 0x55863f967520 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.554+0000 7f71f87eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55863f96f640 con 0x55863f966b30 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33712/0 (socket says 172.21.15.90:33712) 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 -- 172.21.15.90:0/1389669793 learned_addr learned my addr 172.21.15.90:0/1389669793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71ef400640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 0x55863f977e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71e6000640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f970e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 -- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 msgr2=0x55863f970e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f970e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 -- 172.21.15.90:0/1389669793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 msgr2=0x55863f977e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 0x55863f977e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71eea00640 1 -- 172.21.15.90:0/1389669793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863f5db210 con 0x55863f966b30 2024-06-02T15:45:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.555+0000 7f71ef400640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 0x55863f977e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:45:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.556+0000 7f71eea00640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71e000ed00 tx=0x7f71e0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93343cd3483c322d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.556+0000 7f71dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.556+0000 7f71dcc00640 1 -- 172.21.15.90:0/1389669793 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e001db20 con 0x55863f966b30 2024-06-02T15:45:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.556+0000 7f71dcc00640 1 -- 172.21.15.90:0/1389669793 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71e001dcc0 con 0x55863f966b30 2024-06-02T15:45:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.563+0000 7f71f87eb740 10 client.0 _ll_register_callbacks cb 0x55863f84c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:55 smithi135 ceph-mon[36839]: pgmap v558: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 9 op/s 2024-06-02T15:45:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.591+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e8011db0 con 0x55863f966b30 2024-06-02T15:45:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.592+0000 7f71dcc00640 1 -- 172.21.15.90:0/1389669793 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71e0004030 con 0x55863f966b30 2024-06-02T15:45:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.592+0000 7f71f87eb740 10 client.25525 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.592+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55863f6a7620 con 0x55863f966b30 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71dcc00640 1 -- 172.21.15.90:0/1389669793 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71e00041d0 con 0x55863f966b30 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 20 client.25525 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f97c1c0 0x55863f97e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 4 client.25525 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:45:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.607+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863f8d8ae0 con 0x55863f97c1c0 2024-06-02T15:45:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.608+0000 7f71e6000640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f97c1c0 0x55863f97e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.609+0000 7f71e6000640 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f97c1c0 0x55863f97e660 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffe117802658f0e server_cookie=a338e07cc9534e1a in_seq=0 out_seq=0 2024-06-02T15:45:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.609+0000 7f71dcc00640 10 client.25525 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.610+0000 7f71dcc00640 1 -- 172.21.15.90:0/1389669793 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55863f8d8ae0 con 0x55863f97c1c0 2024-06-02T15:45:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.610+0000 7f71dcc00640 10 client.25525 handle_command_reply: tid=0 2024-06-02T15:45:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.610+0000 7f71f87eb740 1 client.25525 shutdown 2024-06-02T15:45:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.610+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 msgr2=0x55863f9708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.610+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71e000ed00 tx=0x7f71e0004730 comp rx=0 tx=0).stop 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 shutdown_connections 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f97c1c0 0x55863f97e660 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f96c690 0x55863f977e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f970e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 --2- 172.21.15.90:0/1389669793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f966b30 0x55863f9708c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 >> 172.21.15.90:0/1389669793 conn(0x55863f9621a0 msgr2=0x55863f961310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71dcc00640 0 client.25525 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 shutdown_connections 2024-06-02T15:45:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 1 -- 172.21.15.90:0/1389669793 wait complete. 2024-06-02T15:45:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.611+0000 7f71f87eb740 20 client.25525 trim_cache size 0 max 16384 2024-06-02T15:45:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 Processor -- start 2024-06-02T15:45:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 -- start start 2024-06-02T15:45:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f9704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f965940 0x55863f970a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b4440 con 0x55863f967520 2024-06-02T15:45:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b1d10 con 0x55863f965940 2024-06-02T15:45:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.643+0000 7f71f87eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01afb60 con 0x55863f96c690 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71eea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f9704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33724/0 (socket says 172.21.15.90:33724) 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 -- 172.21.15.90:0/227623866 learned_addr learned my addr 172.21.15.90:0/227623866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71ef400640 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f965940 0x55863f970a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 -- 172.21.15.90:0/227623866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f965940 msgr2=0x55863f970a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f965940 0x55863f970a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 -- 172.21.15.90:0/227623866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 msgr2=0x55863f9704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f9704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.644+0000 7f71e6000640 1 -- 172.21.15.90:0/227623866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863f961fd0 con 0x55863f96c690 2024-06-02T15:45:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.645+0000 7f71e6000640 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71d80ffd20 tx=0x7f71d802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14318d4c31a153fb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.645+0000 7f71df400640 1 -- 172.21.15.90:0/227623866 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80d26a0 con 0x55863f96c690 2024-06-02T15:45:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.645+0000 7f71df400640 1 -- 172.21.15.90:0/227623866 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d80ffe30 con 0x55863f96c690 2024-06-02T15:45:55.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.645+0000 7f71df400640 1 -- 172.21.15.90:0/227623866 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80d2210 con 0x55863f96c690 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 -- 172.21.15.90:0/227623866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 msgr2=0x55863f977e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71d80ffd20 tx=0x7f71d802a400 comp rx=0 tx=0).stop 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 -- 172.21.15.90:0/227623866 shutdown_connections 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f965940 0x55863f970a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f977e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 --2- 172.21.15.90:0/227623866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f967520 0x55863f9704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 -- 172.21.15.90:0/227623866 >> 172.21.15.90:0/227623866 conn(0x55863f9621a0 msgr2=0x55863f962580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 -- 172.21.15.90:0/227623866 shutdown_connections 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.646+0000 7f71f87eb740 1 -- 172.21.15.90:0/227623866 wait complete. 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.647+0000 7f71f87eb740 1 -- start start 2024-06-02T15:45:55.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.647+0000 7f71f87eb740 1 Processor -- start 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f965940 0x55863f975270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f9757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b01d0 con 0x55863f965940 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b1640 con 0x55863f967520 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b08a0 con 0x55863f96c690 2024-06-02T15:45:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71f87eb740 10 client.0 _ll_register_callbacks cb 0x7f71e000e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71eea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f965940 0x55863f975270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33728/0 (socket says 172.21.15.90:33728) 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 -- 172.21.15.90:0/2307636841 learned_addr learned my addr 172.21.15.90:0/2307636841 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71e6000640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f9757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 -- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 msgr2=0x55863f9757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f9757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 -- 172.21.15.90:0/2307636841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f965940 msgr2=0x55863f975270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f965940 0x55863f975270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.648+0000 7f71ef400640 1 -- 172.21.15.90:0/2307636841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863f976cf0 con 0x55863f96c690 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71ef400640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fd0 tx=0x7f71e8004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c53e5c27ed4469d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e802c060 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d80ffe30 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71e80119f0 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71e8033050 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71f87eb740 10 client.25531 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.649+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55863f9780c0 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d80ffe30 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f71e801b4c0 con 0x55863f96c690 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 10 client.25531 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 20 client.25531 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f9e7140 0x55863f9e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 4 client.25531 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:45:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55863f8d22c0 con 0x55863f9e7140 2024-06-02T15:45:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.650+0000 7f71eea00640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f9e7140 0x55863f9e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.651+0000 7f71eea00640 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f9e7140 0x55863f9e9530 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbeb29a4284d0b98 server_cookie=793d0304442ff168 in_seq=0 out_seq=0 2024-06-02T15:45:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.651+0000 7f71dcc00640 10 client.25531 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.675+0000 7f71dcc00640 1 -- 172.21.15.90:0/2307636841 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55863f8d22c0 con 0x55863f9e7140 2024-06-02T15:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.675+0000 7f71dcc00640 10 client.25531 handle_command_reply: tid=0 2024-06-02T15:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.675+0000 7f71f87eb740 1 client.25531 shutdown 2024-06-02T15:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 msgr2=0x55863f976700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fd0 tx=0x7f71e8004ea0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 shutdown_connections 2024-06-02T15:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55863f9e7140 0x55863f9e9530 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55863f96c690 0x55863f976700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55863f967520 0x55863f9757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 --2- 172.21.15.90:0/2307636841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55863f965940 0x55863f975270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71dcc00640 0 client.25531 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 >> 172.21.15.90:0/2307636841 conn(0x55863f9621a0 msgr2=0x55863f9704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 shutdown_connections 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 1 -- 172.21.15.90:0/2307636841 wait complete. 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.676+0000 7f71f87eb740 20 client.25531 trim_cache size 0 max 16384 2024-06-02T15:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.677+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 msgr2=0x7f71d40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.677+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 0x7f71d40b4eb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71e00095f0 tx=0x7f71e0005f90 comp rx=0 tx=0).stop 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.677+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 msgr2=0x7f71f019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.677+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71d8031490 tx=0x7f71d8003c10 comp rx=0 tx=0).stop 2024-06-02T15:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 shutdown_connections 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71d40b29f0 0x7f71d40b4eb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71f010f080 0x7f71f01a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f0076a90 0x7f71f019f750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 --2- 172.21.15.90:0/1149530297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f71f00754b0 0x7f71f019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 >> 172.21.15.90:0/1149530297 conn(0x7f71f00fe040 msgr2=0x7f71f0102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 shutdown_connections 2024-06-02T15:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:45:55.678+0000 7f71f5a00640 1 -- 172.21.15.90:0/1149530297 wait complete. 2024-06-02T15:45:56.505 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:56.505 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:56.805 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:56.805 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:56.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:56.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:56.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 24 -lt 100 ']' 2024-06-02T15:45:56.834 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 24 + 1 2024-06-02T15:45:56.899 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=25 2024-06-02T15:45:56.899 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:56.957 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:56.957 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:56.961 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:56.962 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:56.963 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:57.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:57 smithi090 ceph-mon[31595]: pgmap v559: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-06-02T15:45:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:57 smithi120 ceph-mon[36451]: pgmap v559: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-06-02T15:45:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:57 smithi135 ceph-mon[36839]: pgmap v559: 129 pgs: 129 active+clean; 1.0 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-06-02T15:45:58.738 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:58.738 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:59.171 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:45:59.171 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:45:59.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:59.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:45:59.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 25 -lt 100 ']' 2024-06-02T15:45:59.215 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 25 + 1 2024-06-02T15:45:59.299 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=26 2024-06-02T15:45:59.300 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:45:59.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:45:59 smithi090 ceph-mon[31595]: pgmap v560: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:59.465 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:45:59.466 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:45:59.469 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:45:59.470 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:45:59.471 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:45:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:45:59 smithi120 ceph-mon[36451]: pgmap v560: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:45:59 smithi135 ceph-mon[36839]: pgmap v560: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:45:59.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 -- 172.21.15.90:0/2717134823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 msgr2=0x7fe840112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 --2- 172.21.15.90:0/2717134823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe840112650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b0a0 tx=0x7fe83800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 -- 172.21.15.90:0/2717134823 shutdown_connections 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 --2- 172.21.15.90:0/2717134823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe840112650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 --2- 172.21.15.90:0/2717134823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe84010b780 0x7fe84010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 --2- 172.21.15.90:0/2717134823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe840105780 0x7fe840105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 -- 172.21.15.90:0/2717134823 >> 172.21.15.90:0/2717134823 conn(0x7fe840101270 msgr2=0x7fe840103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 -- 172.21.15.90:0/2717134823 shutdown_connections 2024-06-02T15:45:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.661+0000 7fe847c00700 1 -- 172.21.15.90:0/2717134823 wait complete. 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 Processor -- start 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 -- start start 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 0x7fe84019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe84019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe84010b780 0x7fe8401a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401159d0 con 0x7fe840105780 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe840115850 con 0x7fe840106140 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.663+0000 7fe847c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe840115b50 con 0x7fe84010b780 2024-06-02T15:45:59.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe845200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 0x7fe84019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe84019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe845200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 0x7fe84019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58378/0 (socket says 172.21.15.90:58378) 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe845200700 1 -- 172.21.15.90:0/2342952611 learned_addr learned my addr 172.21.15.90:0/2342952611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe845c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe84010b780 0x7fe8401a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 -- 172.21.15.90:0/2342952611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe84010b780 msgr2=0x7fe8401a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe84010b780 0x7fe8401a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 -- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 msgr2=0x7fe84019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 0x7fe84019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.664+0000 7fe83fe00700 1 -- 172.21.15.90:0/2342952611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8401a0df0 con 0x7fe840106140 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe83fe00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe84019c320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e9f0 tx=0x7fe82c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe83d600700 1 -- 172.21.15.90:0/2342952611 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c004220 con 0x7fe840106140 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe83d600700 1 -- 172.21.15.90:0/2342952611 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c0043c0 con 0x7fe840106140 2024-06-02T15:45:59.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe83d600700 1 -- 172.21.15.90:0/2342952611 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c005020 con 0x7fe840106140 2024-06-02T15:45:59.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8401a10e0 con 0x7fe840106140 2024-06-02T15:45:59.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.665+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401a8960 con 0x7fe840106140 2024-06-02T15:45:59.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.669+0000 7fe83d600700 1 -- 172.21.15.90:0/2342952611 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe82c004750 con 0x7fe840106140 2024-06-02T15:45:59.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.671+0000 7fe83d600700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 0x7fe8280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.671+0000 7fe83d600700 1 -- 172.21.15.90:0/2342952611 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe82c0e2530 con 0x7fe840106140 2024-06-02T15:45:59.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.671+0000 7fe845200700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 0x7fe8280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.672+0000 7fe845200700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 0x7fe8280b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ca0 tx=0x7fe834009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.808+0000 7fe85266fb80 1 Processor -- start 2024-06-02T15:45:59.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.808+0000 7fe85266fb80 1 -- start start 2024-06-02T15:45:59.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.809+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a972a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.809+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.809+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a96d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.809+0000 7fe85266fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f61a8caec0 con 0x55f61a96d1c0 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe85266fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f61a8baa60 con 0x55f61a972690 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe85266fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f61a8beaa0 con 0x55f61a972fb0 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33754/0 (socket says 172.21.15.90:33754) 2024-06-02T15:45:59.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 -- 172.21.15.90:0/28147189 learned_addr learned my addr 172.21.15.90:0/28147189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe845200700 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a972a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe845c00700 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a96d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 -- 172.21.15.90:0/28147189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 msgr2=0x55f61a972a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a972a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 -- 172.21.15.90:0/28147189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 msgr2=0x55f61a96d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.810+0000 7fe83fe00700 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a96d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.811+0000 7fe83fe00700 1 -- 172.21.15.90:0/28147189 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61a8834a0 con 0x55f61a972fb0 2024-06-02T15:45:59.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.811+0000 7fe83fe00700 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0a57c0 tx=0x7fe82c0a9190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5dbf2069e70111c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.811+0000 7fe832000700 1 -- 172.21.15.90:0/28147189 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0a3e60 con 0x55f61a972fb0 2024-06-02T15:45:59.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.811+0000 7fe832000700 1 -- 172.21.15.90:0/28147189 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c0e2030 con 0x55f61a972fb0 2024-06-02T15:45:59.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.811+0000 7fe832000700 1 -- 172.21.15.90:0/28147189 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c01c070 con 0x55f61a972fb0 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.812+0000 7fe85266fb80 1 -- 172.21.15.90:0/28147189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 msgr2=0x55f61a96cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.812+0000 7fe85266fb80 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0a57c0 tx=0x7fe82c0a9190 comp rx=0 tx=0).stop 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 -- 172.21.15.90:0/28147189 shutdown_connections 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a96d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a972a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 --2- 172.21.15.90:0/28147189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a96cba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 -- 172.21.15.90:0/28147189 >> 172.21.15.90:0/28147189 conn(0x55f61a89ac60 msgr2=0x55f61a967710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 -- 172.21.15.90:0/28147189 shutdown_connections 2024-06-02T15:45:59.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 -- 172.21.15.90:0/28147189 wait complete. 2024-06-02T15:45:59.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.813+0000 7fe85266fb80 1 -- start start 2024-06-02T15:45:59.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 Processor -- start 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a978df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a976190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a9766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f61a7c1020 con 0x55f61a96d1c0 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f61a85b7e0 con 0x55f61a972690 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.814+0000 7fe85266fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f61a979a00 con 0x55f61a972fb0 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe85266fb80 10 client.0 _ll_register_callbacks cb 0x55f61a864cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:59.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe845200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a978df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe845200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a978df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58388/0 (socket says 172.21.15.90:58388) 2024-06-02T15:45:59.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe845200700 1 -- 172.21.15.90:0/1270625190 learned_addr learned my addr 172.21.15.90:0/1270625190 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:59.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a976190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe845c00700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a9766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 -- 172.21.15.90:0/1270625190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 msgr2=0x55f61a9766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a9766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 -- 172.21.15.90:0/1270625190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 msgr2=0x55f61a978df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a978df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.815+0000 7fe83fe00700 1 -- 172.21.15.90:0/1270625190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61a80a950 con 0x55f61a972690 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a976190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c800 tx=0x7fe82c0adaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a02e349ff25f451 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe826a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0c2070 con 0x55f61a972690 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe82c0b2110 con 0x55f61a972690 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c0e2030 con 0x55f61a972690 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe82c0a3e60 con 0x55f61a972690 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe85266fb80 10 client.25580 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:59.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.816+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f61a88b0e0 con 0x55f61a972690 2024-06-02T15:45:59.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c0e2030 con 0x55f61a972690 2024-06-02T15:45:59.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe82c0beb90 con 0x55f61a972690 2024-06-02T15:45:59.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 10 client.25580 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:59.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 20 client.25580 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9864d0 0x55f61a9888c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 4 client.25580 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.817+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61a8cce10 con 0x55f61a9864d0 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.818+0000 7fe845200700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9864d0 0x55f61a9888c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.818+0000 7fe845200700 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9864d0 0x55f61a9888c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f0e48c3a98c102 server_cookie=df140b4ebd982726 in_seq=0 out_seq=0 2024-06-02T15:45:59.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.818+0000 7fe826a00700 10 client.25580 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.819+0000 7fe826a00700 1 -- 172.21.15.90:0/1270625190 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f61a8cce10 con 0x55f61a9864d0 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.819+0000 7fe826a00700 10 client.25580 handle_command_reply: tid=0 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.819+0000 7fe85266fb80 1 client.25580 shutdown 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 msgr2=0x55f61a976190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a976190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c800 tx=0x7fe82c0adaf0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 shutdown_connections 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9864d0 0x55f61a9888c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972fb0 0x55f61a9766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a96d1c0 0x55f61a978df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1270625190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a976190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe826a00700 0 client.25580 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 >> 172.21.15.90:0/1270625190 conn(0x55f61a89ac60 msgr2=0x55f61a966df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 shutdown_connections 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 1 -- 172.21.15.90:0/1270625190 wait complete. 2024-06-02T15:45:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.820+0000 7fe85266fb80 20 client.25580 trim_cache size 0 max 16384 2024-06-02T15:45:59.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.844+0000 7fe85266fb80 1 Processor -- start 2024-06-02T15:45:59.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.844+0000 7fe85266fb80 1 -- start start 2024-06-02T15:45:59.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972690 0x55f61a967570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b3ec0 con 0x55f61a972fb0 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b1750 con 0x55f61a96d1c0 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe85266fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401af5a0 con 0x55f61a972690 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:56034/0 (socket says 172.21.15.90:56034) 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 -- 172.21.15.90:0/1744313663 learned_addr learned my addr 172.21.15.90:0/1744313663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845c00700 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972690 0x55f61a967570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 -- 172.21.15.90:0/1744313663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972690 msgr2=0x55f61a967570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972690 0x55f61a967570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 -- 172.21.15.90:0/1744313663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 msgr2=0x55f61a9719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.845+0000 7fe845200700 1 -- 172.21.15.90:0/1744313663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61a972cc0 con 0x55f61a96d1c0 2024-06-02T15:45:59.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.846+0000 7fe845200700 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe834009b60 tx=0x7fe834014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b737445640c9c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.846+0000 7fe831600700 1 -- 172.21.15.90:0/1744313663 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834033430 con 0x55f61a96d1c0 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.846+0000 7fe831600700 1 -- 172.21.15.90:0/1744313663 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8340298a0 con 0x55f61a96d1c0 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.846+0000 7fe831600700 1 -- 172.21.15.90:0/1744313663 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83401c970 con 0x55f61a96d1c0 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 -- 172.21.15.90:0/1744313663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 msgr2=0x55f61a982ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe834009b60 tx=0x7fe834014000 comp rx=0 tx=0).stop 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 -- 172.21.15.90:0/1744313663 shutdown_connections 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a96d1c0 0x55f61a982ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1744313663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a972690 0x55f61a967570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 -- 172.21.15.90:0/1744313663 >> 172.21.15.90:0/1744313663 conn(0x55f61a89ac60 msgr2=0x55f61a9712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 -- 172.21.15.90:0/1744313663 shutdown_connections 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.847+0000 7fe85266fb80 1 -- 172.21.15.90:0/1744313663 wait complete. 2024-06-02T15:45:59.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.848+0000 7fe85266fb80 1 -- start start 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.848+0000 7fe85266fb80 1 Processor -- start 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a98b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401afc10 con 0x55f61a972fb0 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b1080 con 0x55f61a972690 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8401b02e0 con 0x55f61a96d1c0 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe85266fb80 10 client.0 _ll_register_callbacks cb 0x55f61a864cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe845200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe845200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33788/0 (socket says 172.21.15.90:33788) 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe845200700 1 -- 172.21.15.90:0/1757736472 learned_addr learned my addr 172.21.15.90:0/1757736472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:45:59.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe845c00700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.849+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a98b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 -- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 msgr2=0x55f61a98b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a98b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 -- 172.21.15.90:0/1757736472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 msgr2=0x55f61a9f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 -- 172.21.15.90:0/1757736472 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61a9f12b0 con 0x55f61a96d1c0 2024-06-02T15:45:59.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe845200700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8340094f0 tx=0x7fe8340295d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c3528a9bb66c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:45:59.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.850+0000 7fe826a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:45:59.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834040060 con 0x55f61a96d1c0 2024-06-02T15:45:59.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8340298a0 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe834029c60 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe834047050 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe85266fb80 10 client.25549 fetch_fsmap learned FSMap version 26 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f61a9f25e0 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.851+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8340298a0 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe834004910 con 0x55f61a96d1c0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 10 client.25549 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 20 client.25549 populate_metadata read hostname 'smithi090' 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9f2cd0 0x55f61a9f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 4 client.25549 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f61a8c65f0 con 0x55f61a9f2cd0 2024-06-02T15:45:59.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.852+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9f2cd0 0x55f61a9f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:45:59.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.853+0000 7fe83fe00700 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9f2cd0 0x55f61a9f50c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4dbf0d1540fa6b server_cookie=91352e953038168f in_seq=0 out_seq=0 2024-06-02T15:45:59.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.853+0000 7fe826a00700 10 client.25549 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:59.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.854+0000 7fe826a00700 1 -- 172.21.15.90:0/1757736472 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f61a8c65f0 con 0x55f61a9f2cd0 2024-06-02T15:45:59.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.854+0000 7fe826a00700 10 client.25549 handle_command_reply: tid=0 2024-06-02T15:45:59.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.854+0000 7fe85266fb80 1 client.25549 shutdown 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 msgr2=0x55f61a98ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8340094f0 tx=0x7fe8340295d0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 shutdown_connections 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f61a9f2cd0 0x55f61a9f50c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f61a972fb0 0x55f61a9f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61a96d1c0 0x55f61a98ace0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 --2- 172.21.15.90:0/1757736472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f61a972690 0x55f61a98b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 >> 172.21.15.90:0/1757736472 conn(0x55f61a89ac60 msgr2=0x55f61a971b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe826a00700 0 client.25549 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:45:59.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 shutdown_connections 2024-06-02T15:45:59.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 1 -- 172.21.15.90:0/1757736472 wait complete. 2024-06-02T15:45:59.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.855+0000 7fe85266fb80 20 client.25549 trim_cache size 0 max 16384 2024-06-02T15:45:59.854 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "93e91b8d-3b43-427a-b5dc-4c333066b502", 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.856+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 msgr2=0x7fe8280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.856+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 0x7fe8280b4dc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ca0 tx=0x7fe834009260 comp rx=0 tx=0).stop 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.856+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 msgr2=0x7fe84019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:45:59.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.856+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe84019c320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00e9f0 tx=0x7fe82c00ed00 comp rx=0 tx=0).stop 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 shutdown_connections 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe8280b2900 0x7fe8280b4dc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe84010b780 0x7fe8401a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe840105780 0x7fe84019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 --2- 172.21.15.90:0/2342952611 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe840106140 0x7fe84019c320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 >> 172.21.15.90:0/2342952611 conn(0x7fe840101270 msgr2=0x7fe8401017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 shutdown_connections 2024-06-02T15:45:59.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:45:59.857+0000 7fe847c00700 1 -- 172.21.15.90:0/2342952611 wait complete. 2024-06-02T15:46:01.321 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:01.322 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:01 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:46:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:01 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:01 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:46:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:01 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:01 smithi090 ceph-mon[31595]: pgmap v561: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:01 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:46:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:01 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:01 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:46:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:01 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:01 smithi120 ceph-mon[36451]: pgmap v561: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:01 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:46:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:01 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:01 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:46:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:01 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:46:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:01 smithi135 ceph-mon[36839]: pgmap v561: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:01.671 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:01.671 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:01.758 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:01.759 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:01.759 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 26 -lt 100 ']' 2024-06-02T15:46:01.760 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 26 + 1 2024-06-02T15:46:01.849 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=27 2024-06-02T15:46:01.849 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:01.961 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:01.961 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:01.966 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:01.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:01.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:02.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:02 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:46:02.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:02 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:46:02.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:02 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:46:03.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:03 smithi090 ceph-mon[31595]: pgmap v562: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:03 smithi120 ceph-mon[36451]: pgmap v562: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:03 smithi135 ceph-mon[36839]: pgmap v562: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:03.680 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:03.680 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:04.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:04.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:04.336 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:04.336 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:04.336 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 27 -lt 100 ']' 2024-06-02T15:46:04.337 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 27 + 1 2024-06-02T15:46:04.441 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=28 2024-06-02T15:46:04.441 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:04.590 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:04.590 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:04.594 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:04.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:04.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:05.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:46:05.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:05 smithi090 ceph-mon[31595]: pgmap v563: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-06-02T15:46:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:05 smithi120 ceph-mon[36451]: pgmap v563: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-06-02T15:46:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:05 smithi135 ceph-mon[36839]: pgmap v563: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-06-02T15:46:06.297 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:06.297 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:06.607 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:06.607 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:06.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:06.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:06.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 28 -lt 100 ']' 2024-06-02T15:46:06.655 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 28 + 1 2024-06-02T15:46:06.841 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=29 2024-06-02T15:46:06.841 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:06.908 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:06.908 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:06.913 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:06.914 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:06.915 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:07.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:07 smithi090 ceph-mon[31595]: pgmap v564: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-06-02T15:46:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:07 smithi120 ceph-mon[36451]: pgmap v564: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-06-02T15:46:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:07 smithi135 ceph-mon[36839]: pgmap v564: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-06-02T15:46:08.055 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:08.055 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:08.413 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:08.413 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:08.454 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:08.454 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:08.454 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 29 -lt 100 ']' 2024-06-02T15:46:08.455 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 29 + 1 2024-06-02T15:46:08.599 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=30 2024-06-02T15:46:08.599 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:08.679 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:08.679 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:08.683 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:08.684 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:08.685 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:09 smithi120 ceph-mon[36451]: pgmap v565: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-02T15:46:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:09 smithi135 ceph-mon[36839]: pgmap v565: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-02T15:46:09.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:09 smithi090 ceph-mon[31595]: pgmap v565: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-06-02T15:46:10.446 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:10.447 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:10.946 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:10.946 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:10.970 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:10.970 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:10.970 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 30 -lt 100 ']' 2024-06-02T15:46:10.972 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 30 + 1 2024-06-02T15:46:11.141 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=31 2024-06-02T15:46:11.141 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:11.304 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:11.304 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:11.308 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:11.310 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:11.311 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:11 smithi120 ceph-mon[36451]: pgmap v566: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:46:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:11 smithi135 ceph-mon[36839]: pgmap v566: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:46:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:11 smithi090 ceph-mon[31595]: pgmap v566: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:46:12.854 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:12.854 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:13.238 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:13.238 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:13.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:13.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:13.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 31 -lt 100 ']' 2024-06-02T15:46:13.271 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 31 + 1 2024-06-02T15:46:13.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=32 2024-06-02T15:46:13.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:13.391 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:13.391 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:13.395 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:13.396 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:13.397 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:13 smithi120 ceph-mon[36451]: pgmap v567: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:46:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:13 smithi135 ceph-mon[36839]: pgmap v567: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:46:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:13 smithi090 ceph-mon[31595]: pgmap v567: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:46:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 -- 172.21.15.90:0/2811694664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c078ff0 msgr2=0x7f7e8c079890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 --2- 172.21.15.90:0/2811694664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c079890 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ca0 tx=0x7f7e8402a8b0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 -- 172.21.15.90:0/2811694664 shutdown_connections 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 --2- 172.21.15.90:0/2811694664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c079890 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 --2- 172.21.15.90:0/2811694664 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 --2- 172.21.15.90:0/2811694664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c077410 0x7f7e8c077830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 -- 172.21.15.90:0/2811694664 >> 172.21.15.90:0/2811694664 conn(0x7f7e8c100240 msgr2=0x7f7e8c102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.861+0000 7f7e90c00640 1 -- 172.21.15.90:0/2811694664 shutdown_connections 2024-06-02T15:46:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.862+0000 7f7e90c00640 1 -- 172.21.15.90:0/2811694664 wait complete. 2024-06-02T15:46:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.962+0000 7f7e90c00640 1 Processor -- start 2024-06-02T15:46:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.968+0000 7f7e90c00640 1 -- start start 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c077410 0x7f7e8c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c1a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c114c50 con 0x7f7e8c078ff0 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c114ad0 con 0x7f7e8c078650 2024-06-02T15:46:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.969+0000 7f7e90c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c114dd0 con 0x7f7e8c077410 2024-06-02T15:46:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c077410 0x7f7e8c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c1a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59988/0 (socket says 172.21.15.90:59988) 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 -- 172.21.15.90:0/4138847773 learned_addr learned my addr 172.21.15.90:0/4138847773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 -- 172.21.15.90:0/4138847773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c077410 msgr2=0x7f7e8c19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c077410 0x7f7e8c19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 -- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c078ff0 msgr2=0x7f7e8c1a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c1a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.970+0000 7f7e89400640 1 -- 172.21.15.90:0/4138847773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c1a6df0 con 0x7f7e8c078650 2024-06-02T15:46:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.971+0000 7f7e89400640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9f0 tx=0x7f7e7800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.979+0000 7f7e82a00640 1 -- 172.21.15.90:0/4138847773 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800cd80 con 0x7f7e8c078650 2024-06-02T15:46:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.979+0000 7f7e82a00640 1 -- 172.21.15.90:0/4138847773 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780054d0 con 0x7f7e8c078650 2024-06-02T15:46:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.979+0000 7f7e82a00640 1 -- 172.21.15.90:0/4138847773 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78010690 con 0x7f7e8c078650 2024-06-02T15:46:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.979+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1a70e0 con 0x7f7e8c078650 2024-06-02T15:46:14.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.984+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1a75e0 con 0x7f7e8c078650 2024-06-02T15:46:14.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.986+0000 7f7e82a00640 1 -- 172.21.15.90:0/4138847773 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e78004e90 con 0x7f7e8c078650 2024-06-02T15:46:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.989+0000 7f7e82a00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 0x7f7e700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.989+0000 7f7e82a00640 1 -- 172.21.15.90:0/4138847773 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7e780e6060 con 0x7f7e8c078650 2024-06-02T15:46:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.989+0000 7f7e89e00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 0x7f7e700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:14.991+0000 7f7e89e00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 0x7f7e700b4eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0095f0 tx=0x7f7e7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:15.088 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:15.088 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.491+0000 7f7e939e2740 1 Processor -- start 2024-06-02T15:46:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.492+0000 7f7e939e2740 1 -- start start 2024-06-02T15:46:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.492+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.492+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5ee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e939e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55facc558b90 con 0x55facc5e9520 2024-06-02T15:46:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e939e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55facc548730 con 0x55facc5ee690 2024-06-02T15:46:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e939e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55facc54c770 con 0x55facc5e8b30 2024-06-02T15:46:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57454/0 (socket says 172.21.15.90:57454) 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e89e00640 1 -- 172.21.15.90:0/2488738915 learned_addr learned my addr 172.21.15.90:0/2488738915 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e89400640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5ee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e8a800640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.493+0000 7f7e89e00640 1 -- 172.21.15.90:0/2488738915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 msgr2=0x55facc5f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89e00640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89e00640 1 -- 172.21.15.90:0/2488738915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 msgr2=0x55facc5ee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89e00640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5ee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89e00640 1 -- 172.21.15.90:0/2488738915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facc4e18b0 con 0x55facc5e8b30 2024-06-02T15:46:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89400640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:46:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e89e00640 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0097e0 tx=0x7f7e7c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5eadba4447fc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e77000640 1 -- 172.21.15.90:0/2488738915 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c004020 con 0x55facc5e8b30 2024-06-02T15:46:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e77000640 1 -- 172.21.15.90:0/2488738915 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c0041c0 con 0x55facc5e8b30 2024-06-02T15:46:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.494+0000 7f7e77000640 1 -- 172.21.15.90:0/2488738915 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0267b0 con 0x55facc5e8b30 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.495+0000 7f7e939e2740 1 -- 172.21.15.90:0/2488738915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 msgr2=0x55facc5e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.495+0000 7f7e939e2740 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0097e0 tx=0x7f7e7c014000 comp rx=0 tx=0).stop 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 -- 172.21.15.90:0/2488738915 shutdown_connections 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5ee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 --2- 172.21.15.90:0/2488738915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5e8f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 -- 172.21.15.90:0/2488738915 >> 172.21.15.90:0/2488738915 conn(0x55facc5e41a0 msgr2=0x55facc5e45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 -- 172.21.15.90:0/2488738915 shutdown_connections 2024-06-02T15:46:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.496+0000 7f7e939e2740 1 -- 172.21.15.90:0/2488738915 wait complete. 2024-06-02T15:46:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.529+0000 7f7e939e2740 1 -- start start 2024-06-02T15:46:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.553+0000 7f7e939e2740 1 Processor -- start 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55facc22e790 con 0x55facc5e9520 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55facc4f0bc0 con 0x55facc5e8b30 2024-06-02T15:46:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e939e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55facc5f1640 con 0x55facc5ee690 2024-06-02T15:46:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59996/0 (socket says 172.21.15.90:59996) 2024-06-02T15:46:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e89e00640 1 -- 172.21.15.90:0/3041339083 learned_addr learned my addr 172.21.15.90:0/3041339083 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e8a800640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.554+0000 7f7e89400640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 -- 172.21.15.90:0/3041339083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 msgr2=0x55facc5f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 -- 172.21.15.90:0/3041339083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 msgr2=0x55facc5f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 -- 172.21.15.90:0/3041339083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facc25d210 con 0x55facc5e8b30 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e89e00640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099c0 tx=0x7f7e7c004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f743aa4719d7fbe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.555+0000 7f7e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.556+0000 7f7e6b400640 1 -- 172.21.15.90:0/3041339083 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c034060 con 0x55facc5e8b30 2024-06-02T15:46:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.556+0000 7f7e6b400640 1 -- 172.21.15.90:0/3041339083 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7e7c0264f0 con 0x55facc5e8b30 2024-06-02T15:46:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:15 smithi120 ceph-mon[36451]: pgmap v568: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-06-02T15:46:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.567+0000 7f7e939e2740 10 client.0 _ll_register_callbacks cb 0x55facc4ce400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:15 smithi135 ceph-mon[36839]: pgmap v568: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-06-02T15:46:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.572+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c0041c0 con 0x55facc5e8b30 2024-06-02T15:46:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.572+0000 7f7e6b400640 1 -- 172.21.15.90:0/3041339083 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c0266f0 con 0x55facc5e8b30 2024-06-02T15:46:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.572+0000 7f7e939e2740 10 client.25598 fetch_fsmap learned FSMap version 26 2024-06-02T15:46:15.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.573+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55facc329620 con 0x55facc5e8b30 2024-06-02T15:46:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.608+0000 7f7e6b400640 1 -- 172.21.15.90:0/3041339083 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7e7c02e030 con 0x55facc5e8b30 2024-06-02T15:46:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.608+0000 7f7e939e2740 10 client.25598 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:46:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.608+0000 7f7e939e2740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.608+0000 7f7e939e2740 20 client.25598 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.608+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc5fde40 0x55facc600230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.609+0000 7f7e939e2740 4 client.25598 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:46:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.609+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55facc55aae0 con 0x55facc5fde40 2024-06-02T15:46:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.609+0000 7f7e89400640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc5fde40 0x55facc600230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.610+0000 7f7e89400640 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc5fde40 0x55facc600230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98e3a61eb00d872 server_cookie=7e1eb6522c2eb4f6 in_seq=0 out_seq=0 2024-06-02T15:46:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.610+0000 7f7e6b400640 10 client.25598 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.611+0000 7f7e6b400640 1 -- 172.21.15.90:0/3041339083 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55facc55aae0 con 0x55facc5fde40 2024-06-02T15:46:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.611+0000 7f7e6b400640 10 client.25598 handle_command_reply: tid=0 2024-06-02T15:46:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 client.25598 shutdown 2024-06-02T15:46:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 msgr2=0x55facc5f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099c0 tx=0x7f7e7c004100 comp rx=0 tx=0).stop 2024-06-02T15:46:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 shutdown_connections 2024-06-02T15:46:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc5fde40 0x55facc600230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e6b400640 0 client.25598 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3041339083 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e8b30 0x55facc5f28f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.612+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 >> 172.21.15.90:0/3041339083 conn(0x55facc5e41a0 msgr2=0x55facc5e3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.613+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 shutdown_connections 2024-06-02T15:46:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.613+0000 7f7e939e2740 1 -- 172.21.15.90:0/3041339083 wait complete. 2024-06-02T15:46:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.613+0000 7f7e939e2740 20 client.25598 trim_cache size 0 max 16384 2024-06-02T15:46:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:15 smithi090 ceph-mon[31595]: pgmap v568: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 10 op/s 2024-06-02T15:46:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.661+0000 7f7e939e2740 1 Processor -- start 2024-06-02T15:46:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.661+0000 7f7e939e2740 1 -- start start 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b4530 con 0x55facc5e8b30 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b1d90 con 0x55facc5e9520 2024-06-02T15:46:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.662+0000 7f7e939e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1afbe0 con 0x55facc5ee690 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59998/0 (socket says 172.21.15.90:59998) 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56726/0 (socket says 172.21.15.90:56726) 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89400640 1 -- 172.21.15.90:0/3012712034 learned_addr learned my addr 172.21.15.90:0/3012712034 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e8a800640 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89400640 1 -- 172.21.15.90:0/3012712034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 msgr2=0x55facc5e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.663+0000 7f7e89400640 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e89400640 1 -- 172.21.15.90:0/3012712034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 msgr2=0x55facc5e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e89400640 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e89400640 1 -- 172.21.15.90:0/3012712034 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facc5e8770 con 0x55facc5e9520 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e89e00640 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:46:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e89400640 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9100 tx=0x7f7e780e4000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7beb3dd632a66adc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.664+0000 7f7e76600640 1 -- 172.21.15.90:0/3012712034 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780a8f10 con 0x55facc5e9520 2024-06-02T15:46:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.665+0000 7f7e76600640 1 -- 172.21.15.90:0/3012712034 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780c2070 con 0x55facc5e9520 2024-06-02T15:46:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.665+0000 7f7e76600640 1 -- 172.21.15.90:0/3012712034 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780df9f0 con 0x55facc5e9520 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.665+0000 7f7e939e2740 1 -- 172.21.15.90:0/3012712034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 msgr2=0x55facc5f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.665+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e780a9100 tx=0x7f7e780e4000 comp rx=0 tx=0).stop 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 -- 172.21.15.90:0/3012712034 shutdown_connections 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5ee690 0x55facc5e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5e9520 0x55facc5f9e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 --2- 172.21.15.90:0/3012712034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e8b30 0x55facc5e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 -- 172.21.15.90:0/3012712034 >> 172.21.15.90:0/3012712034 conn(0x55facc5e41a0 msgr2=0x55facc5f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 -- 172.21.15.90:0/3012712034 shutdown_connections 2024-06-02T15:46:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 -- 172.21.15.90:0/3012712034 wait complete. 2024-06-02T15:46:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.666+0000 7f7e939e2740 1 -- start start 2024-06-02T15:46:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 Processor -- start 2024-06-02T15:46:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b0250 con 0x55facc5e9520 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b16c0 con 0x55facc5ee690 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1b0920 con 0x55facc5e8b30 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.667+0000 7f7e939e2740 10 client.0 _ll_register_callbacks cb 0x7f7e7c00e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e89400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60010/0 (socket says 172.21.15.90:60010) 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 -- 172.21.15.90:0/1187449778 learned_addr learned my addr 172.21.15.90:0/1187449778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e89400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56742/0 (socket says 172.21.15.90:56742) 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e89e00640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 -- 172.21.15.90:0/1187449778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 msgr2=0x55facc5f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 -- 172.21.15.90:0/1187449778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 msgr2=0x55facc5f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 -- 172.21.15.90:0/1187449778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facc5f8b40 con 0x55facc5ee690 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e89e00640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e89400640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e8a800640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ca0 tx=0x7f7e84030ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e5b4ce0bec6614 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.668+0000 7f7e6b400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84003d80 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7e84004040 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780c2070 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e84055050 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e939e2740 10 client.25604 fetch_fsmap learned FSMap version 26 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.669+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55facc5f9ec0 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e780c2070 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7e84050030 con 0x55facc5ee690 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 10 client.25604 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 20 client.25604 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc668de0 0x55facc66b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 4 client.25604 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:46:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55facc5542c0 con 0x55facc668de0 2024-06-02T15:46:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.670+0000 7f7e89e00640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc668de0 0x55facc66b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.671+0000 7f7e89e00640 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc668de0 0x55facc66b200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94a652c614a73bd server_cookie=3d65a797c425fc37 in_seq=0 out_seq=0 2024-06-02T15:46:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.671+0000 7f7e6b400640 10 client.25604 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:15.687 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:15.687 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e6b400640 1 -- 172.21.15.90:0/1187449778 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55facc5542c0 con 0x55facc668de0 2024-06-02T15:46:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e6b400640 10 client.25604 handle_command_reply: tid=0 2024-06-02T15:46:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 client.25604 shutdown 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 msgr2=0x55facc5f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ca0 tx=0x7f7e84030ba0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 shutdown_connections 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55facc668de0 0x55facc66b200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55facc5ee690 0x55facc5f8550 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e6b400640 0 client.25604 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55facc5e9520 0x55facc5f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.691+0000 7f7e939e2740 1 --2- 172.21.15.90:0/1187449778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55facc5e8b30 0x55facc5f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.692+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 >> 172.21.15.90:0/1187449778 conn(0x55facc5e41a0 msgr2=0x55facc5ed660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.692+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 shutdown_connections 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.692+0000 7f7e939e2740 1 -- 172.21.15.90:0/1187449778 wait complete. 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.692+0000 7f7e939e2740 20 client.25604 trim_cache size 0 max 16384 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:46:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:46:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:46:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 msgr2=0x7f7e700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 0x7f7e700b4eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0095f0 tx=0x7f7e7c005f90 comp rx=0 tx=0).stop 2024-06-02T15:46:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 msgr2=0x7f7e8c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800e9f0 tx=0x7f7e7800edf0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 shutdown_connections 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7e700b29f0 0x7f7e700b4eb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e8c078ff0 0x7f7e8c1a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e8c078650 0x7f7e8c19f760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 --2- 172.21.15.90:0/4138847773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e8c077410 0x7f7e8c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.693+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 >> 172.21.15.90:0/4138847773 conn(0x7f7e8c100240 msgr2=0x7f7e8c101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.694+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 shutdown_connections 2024-06-02T15:46:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:15.694+0000 7f7e90c00640 1 -- 172.21.15.90:0/4138847773 wait complete. 2024-06-02T15:46:15.783 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:15.783 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:15.783 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 32 -lt 100 ']' 2024-06-02T15:46:15.785 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 32 + 1 2024-06-02T15:46:15.880 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=33 2024-06-02T15:46:15.881 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:16.042 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:16.042 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:16.045 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:16.046 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:16.046 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:17.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:17 smithi090 ceph-mon[31595]: pgmap v569: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:46:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:17 smithi120 ceph-mon[36451]: pgmap v569: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:46:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:17 smithi135 ceph-mon[36839]: pgmap v569: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 11 op/s 2024-06-02T15:46:17.704 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:17.704 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:18.121 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:18.122 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:18.167 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:18.167 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:18.167 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 33 -lt 100 ']' 2024-06-02T15:46:18.169 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 33 + 1 2024-06-02T15:46:18.274 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=34 2024-06-02T15:46:18.274 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:18.401 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:18.401 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:18.406 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:18.407 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:18.408 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:19.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:19 smithi090 ceph-mon[31595]: pgmap v570: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:46:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:19 smithi120 ceph-mon[36451]: pgmap v570: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:46:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:19 smithi135 ceph-mon[36839]: pgmap v570: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:46:19.796 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:19.796 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:20.196 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:20.196 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:20.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:20.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:20.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 34 -lt 100 ']' 2024-06-02T15:46:20.241 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 34 + 1 2024-06-02T15:46:20.399 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=35 2024-06-02T15:46:20.399 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:20.510 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:20.510 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:20.515 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:20.516 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:20.517 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:21.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:21 smithi090 ceph-mon[31595]: pgmap v571: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-06-02T15:46:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:21 smithi120 ceph-mon[36451]: pgmap v571: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-06-02T15:46:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:21 smithi135 ceph-mon[36839]: pgmap v571: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-06-02T15:46:21.912 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:21.912 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:22.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:22.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:22.274 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:22.274 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:22.274 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 35 -lt 100 ']' 2024-06-02T15:46:22.276 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 35 + 1 2024-06-02T15:46:22.383 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=36 2024-06-02T15:46:22.384 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:22.417 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:22.417 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:22.421 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:22.422 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:22.423 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:23.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:23 smithi090 ceph-mon[31595]: pgmap v572: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:46:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:23 smithi120 ceph-mon[36451]: pgmap v572: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:46:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:23 smithi135 ceph-mon[36839]: pgmap v572: 129 pgs: 129 active+clean; 1.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:46:23.971 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:23.971 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:24.420 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:24.420 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:24.465 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:24.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:24.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 36 -lt 100 ']' 2024-06-02T15:46:24.467 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 36 + 1 2024-06-02T15:46:24.546 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=37 2024-06-02T15:46:24.546 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:24.600 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:24.600 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:24.605 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:24.605 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:24.606 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:25 smithi120 ceph-mon[36451]: pgmap v573: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:25 smithi135 ceph-mon[36839]: pgmap v573: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:25 smithi090 ceph-mon[31595]: pgmap v573: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:26.129 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:26.129 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:26.421 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:26.421 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:26.496 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:26.497 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:26.497 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 37 -lt 100 ']' 2024-06-02T15:46:26.498 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 37 + 1 2024-06-02T15:46:26.623 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=38 2024-06-02T15:46:26.624 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:26.734 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:26.734 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:26.738 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:26.739 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:26.740 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:27 smithi120 ceph-mon[36451]: pgmap v574: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:27 smithi135 ceph-mon[36839]: pgmap v574: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:27 smithi090 ceph-mon[31595]: pgmap v574: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:46:28.137 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:28.137 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:28.554 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:28.554 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:28.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:28.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:28.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 38 -lt 100 ']' 2024-06-02T15:46:28.648 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 38 + 1 2024-06-02T15:46:28.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=39 2024-06-02T15:46:28.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:28.842 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:28.842 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:28.847 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:28.847 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:28.848 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:29 smithi120 ceph-mon[36451]: pgmap v575: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:46:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:29 smithi135 ceph-mon[36839]: pgmap v575: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:46:29.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:29 smithi090 ceph-mon[31595]: pgmap v575: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-06-02T15:46:30.278 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:30.279 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:30.629 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:30.629 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:30.657 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:30.657 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:30.657 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 39 -lt 100 ']' 2024-06-02T15:46:30.658 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 39 + 1 2024-06-02T15:46:30.798 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=40 2024-06-02T15:46:30.798 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:30.833 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:30.834 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:30.838 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:30.838 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:30.839 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:31 smithi120 ceph-mon[36451]: pgmap v576: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:46:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:31 smithi135 ceph-mon[36839]: pgmap v576: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:46:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:31 smithi090 ceph-mon[31595]: pgmap v576: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:46:32.604 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:32.605 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:33.087 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:33.087 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:33.110 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:33.111 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:33.111 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 40 -lt 100 ']' 2024-06-02T15:46:33.111 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 40 + 1 2024-06-02T15:46:33.329 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=41 2024-06-02T15:46:33.329 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:33.394 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:33.395 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:33.400 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:33.401 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:33.401 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:33 smithi120 ceph-mon[36451]: pgmap v577: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-02T15:46:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:33 smithi135 ceph-mon[36839]: pgmap v577: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-02T15:46:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:33 smithi090 ceph-mon[31595]: pgmap v577: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 7 op/s 2024-06-02T15:46:35.162 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:35.162 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:35 smithi120 ceph-mon[36451]: pgmap v578: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:46:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:35 smithi135 ceph-mon[36839]: pgmap v578: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:46:35.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:35.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:35 smithi090 ceph-mon[31595]: pgmap v578: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:46:35.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:35.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:35.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 41 -lt 100 ']' 2024-06-02T15:46:35.651 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 41 + 1 2024-06-02T15:46:35.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=42 2024-06-02T15:46:35.765 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:36.000 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:36.001 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:36.005 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:36.006 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:36.007 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.013+0000 7f234ea00640 1 -- 172.21.15.90:0/983707980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 msgr2=0x7f2348104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.013+0000 7f234ea00640 1 --2- 172.21.15.90:0/983707980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f2348104f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f233c009900 tx=0x7f233c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 -- 172.21.15.90:0/983707980 shutdown_connections 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 --2- 172.21.15.90:0/983707980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 0x7f234810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 --2- 172.21.15.90:0/983707980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f2348106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 --2- 172.21.15.90:0/983707980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f2348104f30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 -- 172.21.15.90:0/983707980 >> 172.21.15.90:0/983707980 conn(0x7f2348100080 msgr2=0x7f23481024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 -- 172.21.15.90:0/983707980 shutdown_connections 2024-06-02T15:46:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.014+0000 7f234ea00640 1 -- 172.21.15.90:0/983707980 wait complete. 2024-06-02T15:46:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.186+0000 7f234ea00640 1 Processor -- start 2024-06-02T15:46:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.186+0000 7f234ea00640 1 -- start start 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.187+0000 7f234ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.187+0000 7f234ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f234819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.187+0000 7f234ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 0x7f23481a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110690 con 0x7f2348105d00 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234ea00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110510 con 0x7f2348104b10 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234ea00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2348110810 con 0x7f23481066c0 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35604/0 (socket says 172.21.15.90:35604) 2024-06-02T15:46:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 -- 172.21.15.90:0/3711500256 learned_addr learned my addr 172.21.15.90:0/3711500256 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234cc00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f234819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234e000640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 0x7f23481a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 -- 172.21.15.90:0/3711500256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 msgr2=0x7f23481a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 0x7f23481a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 -- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 msgr2=0x7f234819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f234819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234d600640 1 -- 172.21.15.90:0/3711500256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23481a2940 con 0x7f2348104b10 2024-06-02T15:46:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.188+0000 7f234cc00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f234819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:46:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.189+0000 7f234d600640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f233c031500 tx=0x7f233c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.199+0000 7f2346000640 1 -- 172.21.15.90:0/3711500256 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c03f070 con 0x7f2348104b10 2024-06-02T15:46:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.199+0000 7f2346000640 1 -- 172.21.15.90:0/3711500256 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c031c50 con 0x7f2348104b10 2024-06-02T15:46:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.199+0000 7f2346000640 1 -- 172.21.15.90:0/3711500256 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c035030 con 0x7f2348104b10 2024-06-02T15:46:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.199+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481a2bd0 con 0x7f2348104b10 2024-06-02T15:46:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.204+0000 7f2346000640 1 -- 172.21.15.90:0/3711500256 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f233c042070 con 0x7f2348104b10 2024-06-02T15:46:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.207+0000 7f2346000640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 0x7f23300b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.208+0000 7f234cc00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 0x7f23300b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.209+0000 7f234cc00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 0x7f23300b4df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2334005d00 tx=0x7f2334005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.222+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481a3070 con 0x7f2348104b10 2024-06-02T15:46:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.223+0000 7f2346000640 1 -- 172.21.15.90:0/3711500256 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f233c0c5de0 con 0x7f2348104b10 2024-06-02T15:46:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.495+0000 7f2352e73740 1 Processor -- start 2024-06-02T15:46:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.495+0000 7f2352e73740 1 -- start start 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a4b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a6d29bab90 con 0x55a6d2a4b6e0 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6d29aa730 con 0x55a6d2a4acf0 2024-06-02T15:46:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f2352e73740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6d29ae770 con 0x55a6d2a50850 2024-06-02T15:46:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f234e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f234d600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a4b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f234e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:51462/0 (socket says 172.21.15.90:51462) 2024-06-02T15:46:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.496+0000 7f234e000640 1 -- 172.21.15.90:0/727386661 learned_addr learned my addr 172.21.15.90:0/727386661 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234cc00640 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 -- 172.21.15.90:0/727386661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 msgr2=0x55a6d2a52ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 -- 172.21.15.90:0/727386661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 msgr2=0x55a6d2a501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 -- 172.21.15.90:0/727386661 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d29438b0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234e000640 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:46:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.497+0000 7f234d600640 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a4b110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23481a2da0 tx=0x7f233c0c90b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cdad41031f1e70 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.498+0000 7f233a600640 1 -- 172.21.15.90:0/727386661 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0e5020 con 0x55a6d2a4acf0 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.498+0000 7f233a600640 1 -- 172.21.15.90:0/727386661 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c0c9880 con 0x55a6d2a4acf0 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.498+0000 7f233a600640 1 -- 172.21.15.90:0/727386661 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0d24b0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 -- 172.21.15.90:0/727386661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 msgr2=0x55a6d2a4b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a4b110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23481a2da0 tx=0x7f233c0c90b0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 -- 172.21.15.90:0/727386661 shutdown_connections 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a52ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 --2- 172.21.15.90:0/727386661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a4b110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 -- 172.21.15.90:0/727386661 >> 172.21.15.90:0/727386661 conn(0x55a6d2a46360 msgr2=0x55a6d2a46760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 -- 172.21.15.90:0/727386661 shutdown_connections 2024-06-02T15:46:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.500+0000 7f2352e73740 1 -- 172.21.15.90:0/727386661 wait complete. 2024-06-02T15:46:36.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.542+0000 7f2352e73740 1 -- start start 2024-06-02T15:46:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.560+0000 7f2352e73740 1 Processor -- start 2024-06-02T15:46:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.561+0000 7f2352e73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f2352e73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a54cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f2352e73740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f2352e73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a6d2690790 con 0x55a6d2a4b6e0 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f2352e73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a6d2952bc0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f2352e73740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6d2a535c0 con 0x55a6d2a50850 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f234cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a54cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f234e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f234cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a54cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36508/0 (socket says 172.21.15.90:36508) 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f234cc00640 1 -- 172.21.15.90:0/1693915239 learned_addr learned my addr 172.21.15.90:0/1693915239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.562+0000 7f234d600640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 -- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 msgr2=0x55a6d2a547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 -- 172.21.15.90:0/1693915239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 msgr2=0x55a6d2a54cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a54cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 -- 172.21.15.90:0/1693915239 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d26bf210 con 0x55a6d2a50850 2024-06-02T15:46:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f234e000640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a5bd30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55a6d2a47230 tx=0x7f234000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7627fe529eed643 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f232ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:46:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.563+0000 7f2352e73740 10 client.0 _ll_register_callbacks cb 0x55a6d2930400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.564+0000 7f232ea00640 1 -- 172.21.15.90:0/1693915239 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340009e30 con 0x55a6d2a50850 2024-06-02T15:46:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.564+0000 7f232ea00640 1 -- 172.21.15.90:0/1693915239 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2340003970 con 0x55a6d2a50850 2024-06-02T15:46:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.569+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f233c0c9880 con 0x55a6d2a50850 2024-06-02T15:46:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.573+0000 7f232ea00640 1 -- 172.21.15.90:0/1693915239 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2340025050 con 0x55a6d2a50850 2024-06-02T15:46:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.573+0000 7f2352e73740 10 client.25579 fetch_fsmap learned FSMap version 26 2024-06-02T15:46:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.573+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a6d278b620 con 0x55a6d2a50850 2024-06-02T15:46:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f232ea00640 1 -- 172.21.15.90:0/1693915239 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2340019070 con 0x55a6d2a50850 2024-06-02T15:46:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 10 client.25579 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:46:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 20 client.25579 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2a600e0 0x55a6d2a62570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 4 client.25579 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:46:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.606+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6d29bcae0 con 0x55a6d2a600e0 2024-06-02T15:46:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.607+0000 7f234d600640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2a600e0 0x55a6d2a62570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.608+0000 7f234d600640 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2a600e0 0x55a6d2a62570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b2efa2aa5b7125 server_cookie=348934a0e59852ba in_seq=0 out_seq=0 2024-06-02T15:46:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.608+0000 7f232ea00640 10 client.25579 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.609+0000 7f232ea00640 1 -- 172.21.15.90:0/1693915239 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a6d29bcae0 con 0x55a6d2a600e0 2024-06-02T15:46:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.609+0000 7f232ea00640 10 client.25579 handle_command_reply: tid=0 2024-06-02T15:46:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 client.25579 shutdown 2024-06-02T15:46:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 msgr2=0x55a6d2a5bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a5bd30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55a6d2a47230 tx=0x7f234000f790 comp rx=0 tx=0).stop 2024-06-02T15:46:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 shutdown_connections 2024-06-02T15:46:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2a600e0 0x55a6d2a62570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a50850 0x55a6d2a5bd30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a54cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f232ea00640 0 client.25579 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 --2- 172.21.15.90:0/1693915239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.610+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 >> 172.21.15.90:0/1693915239 conn(0x55a6d2a46360 msgr2=0x55a6d2a454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.611+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 shutdown_connections 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.611+0000 7f2352e73740 1 -- 172.21.15.90:0/1693915239 wait complete. 2024-06-02T15:46:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.611+0000 7f2352e73740 20 client.25579 trim_cache size 0 max 16384 2024-06-02T15:46:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.658+0000 7f2352e73740 1 Processor -- start 2024-06-02T15:46:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.658+0000 7f2352e73740 1 -- start start 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a5bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a50850 0x55a6d2a45340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f23481b0040 con 0x55a6d2a4b6e0 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ad8a0 con 0x55a6d2a50850 2024-06-02T15:46:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.659+0000 7f2352e73740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ab6f0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a5bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a5bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36510/0 (socket says 172.21.15.90:36510) 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234cc00640 1 -- 172.21.15.90:0/3937761415 learned_addr learned my addr 172.21.15.90:0/3937761415 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:51486/0 (socket says 172.21.15.90:51486) 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234e000640 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a50850 0x55a6d2a45340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 -- 172.21.15.90:0/3937761415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a50850 msgr2=0x55a6d2a45340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a50850 0x55a6d2a45340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 -- 172.21.15.90:0/3937761415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 msgr2=0x55a6d2a5bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a5bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 -- 172.21.15.90:0/3937761415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6d2a55650 con 0x55a6d2a4acf0 2024-06-02T15:46:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.660+0000 7f234d600640 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55a6d2a53450 tx=0x7f233c0d2170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5744802ec7667ab3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.661+0000 7f2339c00640 1 -- 172.21.15.90:0/3937761415 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0e5040 con 0x55a6d2a4acf0 2024-06-02T15:46:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.661+0000 7f2339c00640 1 -- 172.21.15.90:0/3937761415 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c0c5a80 con 0x55a6d2a4acf0 2024-06-02T15:46:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.661+0000 7f2339c00640 1 -- 172.21.15.90:0/3937761415 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0c7a30 con 0x55a6d2a4acf0 2024-06-02T15:46:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 -- 172.21.15.90:0/3937761415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 msgr2=0x55a6d2a49b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55a6d2a53450 tx=0x7f233c0d2170 comp rx=0 tx=0).stop 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 -- 172.21.15.90:0/3937761415 shutdown_connections 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a50850 0x55a6d2a45340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a5bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 --2- 172.21.15.90:0/3937761415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a49b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 -- 172.21.15.90:0/3937761415 >> 172.21.15.90:0/3937761415 conn(0x55a6d2a46360 msgr2=0x55a6d2a44c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 -- 172.21.15.90:0/3937761415 shutdown_connections 2024-06-02T15:46:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.662+0000 7f2352e73740 1 -- 172.21.15.90:0/3937761415 wait complete. 2024-06-02T15:46:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.663+0000 7f2352e73740 1 -- start start 2024-06-02T15:46:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.664+0000 7f2352e73740 1 Processor -- start 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a59df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 0x55a6d2a5ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f23481abd60 con 0x55a6d2a50850 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ad1d0 con 0x55a6d2a4b6e0 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23481ac430 con 0x55a6d2a4acf0 2024-06-02T15:46:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f2352e73740 10 client.0 _ll_register_callbacks cb 0x7f2340009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f234d600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f234d600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:51492/0 (socket says 172.21.15.90:51492) 2024-06-02T15:46:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f234d600640 1 -- 172.21.15.90:0/3475830687 learned_addr learned my addr 172.21.15.90:0/3475830687 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f234e000640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 0x55a6d2a5ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.665+0000 7f234cc00640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a59df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 -- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4b6e0 msgr2=0x55a6d2a59df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a59df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 -- 172.21.15.90:0/3475830687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 msgr2=0x55a6d2a5ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 0x55a6d2a5ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 -- 172.21.15.90:0/3475830687 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6d2a5b3f0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234e000640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 0x55a6d2a5ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f234d600640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f233c005ab0 tx=0x7f233c0ca0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ef88b04c308c11 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f232ea00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c0e5050 con 0x55a6d2a4acf0 2024-06-02T15:46:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f233c0c77d0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.666+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f233c0c5a80 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f233c0d2250 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 10 client.25585 fetch_fsmap learned FSMap version 26 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6d2acb1f0 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c0c5a80 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f233c03b460 con 0x55a6d2a4acf0 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 10 client.25585 fetch_fsmap finished waiting for FSMap version 26 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 20 client.25585 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2acb750 0x55a6d2acdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 4 client.25585 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.667+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6d29b62c0 con 0x55a6d2acb750 2024-06-02T15:46:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.668+0000 7f234cc00640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2acb750 0x55a6d2acdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.669+0000 7f234cc00640 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2acb750 0x55a6d2acdb40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5148dfa449832d07 server_cookie=ff2abfdea28e6b56 in_seq=0 out_seq=0 2024-06-02T15:46:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.669+0000 7f232ea00640 10 client.25585 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.693+0000 7f232ea00640 1 -- 172.21.15.90:0/3475830687 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a6d29b62c0 con 0x55a6d2acb750 2024-06-02T15:46:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.693+0000 7f232ea00640 10 client.25585 handle_command_reply: tid=0 2024-06-02T15:46:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.693+0000 7f2352e73740 1 client.25585 shutdown 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.693+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 msgr2=0x55a6d2a598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.693+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f233c005ab0 tx=0x7f233c0ca0c0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 shutdown_connections 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a6d2acb750 0x55a6d2acdb40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a6d2a50850 0x55a6d2a5ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a6d2a4b6e0 0x55a6d2a59df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 --2- 172.21.15.90:0/3475830687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6d2a4acf0 0x55a6d2a598b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f232ea00640 0 client.25585 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 >> 172.21.15.90:0/3475830687 conn(0x55a6d2a46360 msgr2=0x55a6d2a454c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 shutdown_connections 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 1 -- 172.21.15.90:0/3475830687 wait complete. 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.694+0000 7f2352e73740 20 client.25585 trim_cache size 0 max 16384 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:46:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:46:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 msgr2=0x7f23300b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 0x7f23300b4df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2334005d00 tx=0x7f2334005c70 comp rx=0 tx=0).stop 2024-06-02T15:46:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 msgr2=0x7f234819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f233c031500 tx=0x7f233c003c10 comp rx=0 tx=0).stop 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 shutdown_connections 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f23300b2930 0x7f23300b4df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23481066c0 0x7f23481a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2348105d00 0x7f234819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 --2- 172.21.15.90:0/3711500256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2348104b10 0x7f234819ad30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 >> 172.21.15.90:0/3711500256 conn(0x7f2348100080 msgr2=0x7f2348109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.695+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 shutdown_connections 2024-06-02T15:46:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:36.696+0000 7f234ea00640 1 -- 172.21.15.90:0/3711500256 wait complete. 2024-06-02T15:46:37.545 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:37 smithi090 ceph-mon[31595]: pgmap v579: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-06-02T15:46:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:37 smithi120 ceph-mon[36451]: pgmap v579: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-06-02T15:46:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:37 smithi135 ceph-mon[36839]: pgmap v579: 129 pgs: 129 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 KiB/s wr, 8 op/s 2024-06-02T15:46:37.703 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:37.703 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:38.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:38.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:38.253 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:38.253 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:38.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 42 -lt 100 ']' 2024-06-02T15:46:38.255 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 42 + 1 2024-06-02T15:46:38.398 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=43 2024-06-02T15:46:38.398 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:38.474 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:38.475 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:38.478 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:38.479 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:38.480 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: pgmap v580: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: pgmap v580: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: pgmap v580: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-06-02T15:46:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:46:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:39.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:40.120 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:40.120 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:40 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:46:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:40 smithi120 ceph-mon[36451]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:46:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:40 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:46:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:40 smithi135 ceph-mon[36839]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:46:40.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:40 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:46:40.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:40 smithi090 ceph-mon[31595]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:46:40.653 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:40.653 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:40.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:40.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:40.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 43 -lt 100 ']' 2024-06-02T15:46:40.692 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 43 + 1 2024-06-02T15:46:40.766 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:46:40.806 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=44 2024-06-02T15:46:40.806 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:40.885 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:40.885 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:40.889 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:40.889 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:40.890 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:41.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:41 smithi090 ceph-mon[31595]: pgmap v581: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-06-02T15:46:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:41 smithi120 ceph-mon[36451]: pgmap v581: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-06-02T15:46:41.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:41 smithi135 ceph-mon[36839]: pgmap v581: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-06-02T15:46:42.381 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:42.381 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:42.895 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:42.895 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:42.931 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:42.932 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:42.932 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 44 -lt 100 ']' 2024-06-02T15:46:42.933 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 44 + 1 2024-06-02T15:46:43.038 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=45 2024-06-02T15:46:43.039 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:43.125 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:43.125 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:43.129 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:43.130 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:43.131 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:43 smithi090 ceph-mon[31595]: pgmap v582: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:46:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:43 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:43 smithi120 ceph-mon[36451]: pgmap v582: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:46:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:43 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:43 smithi135 ceph-mon[36839]: pgmap v582: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:46:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:43 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:44.794 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:44.795 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:45.178 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:45.179 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:45.251 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:45.251 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:45.251 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 45 -lt 100 ']' 2024-06-02T15:46:45.252 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 45 + 1 2024-06-02T15:46:45.422 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=46 2024-06-02T15:46:45.423 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:45.502 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:45.503 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:45.507 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:45.508 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:45.509 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:45 smithi090 ceph-mon[31595]: pgmap v583: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-06-02T15:46:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:45 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:45 smithi120 ceph-mon[36451]: pgmap v583: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-06-02T15:46:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:45 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:45.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:45 smithi135 ceph-mon[36839]: pgmap v583: 129 pgs: 129 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-06-02T15:46:45.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:45 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:46:47.061 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:47.062 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:47.786 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:47.786 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:47.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:47 smithi120 ceph-mon[36451]: pgmap v584: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-02T15:46:47.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:47 smithi135 ceph-mon[36839]: pgmap v584: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-02T15:46:47.847 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:47.847 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:47.847 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 46 -lt 100 ']' 2024-06-02T15:46:47.848 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 46 + 1 2024-06-02T15:46:47.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:47 smithi090 ceph-mon[31595]: pgmap v584: 129 pgs: 129 active+clean; 1.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-06-02T15:46:47.956 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=47 2024-06-02T15:46:47.956 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:48.004 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:48.005 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:48.009 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:48.010 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:48.010 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:46:48.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:48 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:46:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:46:48.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:46:48.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:48 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:46:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:46:48.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:48 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:46:49.111 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:49.112 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:49 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:46:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:49 smithi090 ceph-mon[31595]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:46:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:49 smithi090 ceph-mon[31595]: pgmap v585: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:49.628 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:49.628 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:49.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:49.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:49.691 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 47 -lt 100 ']' 2024-06-02T15:46:49.693 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 47 + 1 2024-06-02T15:46:49.764 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=48 2024-06-02T15:46:49.764 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:49.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:49 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:46:49.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:49 smithi120 ceph-mon[36451]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:46:49.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:49 smithi120 ceph-mon[36451]: pgmap v585: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:49.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:49 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:46:49.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:49 smithi135 ceph-mon[36839]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:46:49.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:49 smithi135 ceph-mon[36839]: pgmap v585: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:46:49.868 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:49.868 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:49.872 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:49.873 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:49.875 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:51.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:51 smithi120 ceph-mon[36451]: pgmap v586: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:46:51.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:51 smithi135 ceph-mon[36839]: pgmap v586: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:46:51.844 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:51.845 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:51.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:51 smithi090 ceph-mon[31595]: pgmap v586: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:46:52.156 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:52.157 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:52.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:52.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:52.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 48 -lt 100 ']' 2024-06-02T15:46:52.213 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 48 + 1 2024-06-02T15:46:52.294 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=49 2024-06-02T15:46:52.294 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:52.352 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:52.352 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:52.357 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:52.358 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:52.358 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:53.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:53 smithi090 ceph-mon[31595]: pgmap v587: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-06-02T15:46:53.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:53 smithi120 ceph-mon[36451]: pgmap v587: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-06-02T15:46:53.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:53 smithi135 ceph-mon[36839]: pgmap v587: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 KiB/s wr, 11 op/s 2024-06-02T15:46:54.294 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:54.294 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:55.011 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:55.012 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:55.061 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:55.061 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:55.062 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 49 -lt 100 ']' 2024-06-02T15:46:55.063 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 49 + 1 2024-06-02T15:46:55.147 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=50 2024-06-02T15:46:55.147 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:55.292 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:55.293 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:55.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:55.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:55.298 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:55.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:55 smithi120 ceph-mon[36451]: pgmap v588: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:46:55.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:55 smithi135 ceph-mon[36839]: pgmap v588: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:46:55.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:55 smithi090 ceph-mon[31595]: pgmap v588: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:46:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.858+0000 7f6c57400640 1 -- 172.21.15.90:0/1086784920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50106890 msgr2=0x7f6c5010d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.858+0000 7f6c57400640 1 --2- 172.21.15.90:0/1086784920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50106890 0x7f6c5010d120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c4802a400 comp rx=0 tx=0).stop 2024-06-02T15:46:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 -- 172.21.15.90:0/1086784920 shutdown_connections 2024-06-02T15:46:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 --2- 172.21.15.90:0/1086784920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50106890 0x7f6c5010d120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 --2- 172.21.15.90:0/1086784920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c50106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 --2- 172.21.15.90:0/1086784920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50104ce0 0x7f6c50105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 -- 172.21.15.90:0/1086784920 >> 172.21.15.90:0/1086784920 conn(0x7f6c50100270 msgr2=0x7f6c50102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 -- 172.21.15.90:0/1086784920 shutdown_connections 2024-06-02T15:46:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.859+0000 7f6c57400640 1 -- 172.21.15.90:0/1086784920 wait complete. 2024-06-02T15:46:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.907+0000 7f6c57400640 1 Processor -- start 2024-06-02T15:46:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.907+0000 7f6c57400640 1 -- start start 2024-06-02T15:46:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.908+0000 7f6c57400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 0x7f6c5019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.908+0000 7f6c57400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.908+0000 7f6c57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 0x7f6c501a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c57400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50110860 con 0x7f6c50106890 2024-06-02T15:46:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c57400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501106e0 con 0x7f6c50104ce0 2024-06-02T15:46:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c57400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501109e0 con 0x7f6c50105ed0 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 0x7f6c5019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 0x7f6c5019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:40528/0 (socket says 172.21.15.90:40528) 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41468/0 (socket says 172.21.15.90:41468) 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c54a00640 1 -- 172.21.15.90:0/2265227823 learned_addr learned my addr 172.21.15.90:0/2265227823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c55400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 0x7f6c501a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2265227823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 msgr2=0x7f6c5019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.909+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 0x7f6c5019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.910+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 msgr2=0x7f6c501a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.910+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 0x7f6c501a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.910+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2265227823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501a2a40 con 0x7f6c50105ed0 2024-06-02T15:46:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.910+0000 7f6c55400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 0x7f6c501a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:46:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.910+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009fd0 tx=0x7f6c4000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.916+0000 7f6c4d600640 1 -- 172.21.15.90:0/2265227823 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40004b20 con 0x7f6c50105ed0 2024-06-02T15:46:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.916+0000 7f6c4d600640 1 -- 172.21.15.90:0/2265227823 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c40004cc0 con 0x7f6c50105ed0 2024-06-02T15:46:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.916+0000 7f6c4d600640 1 -- 172.21.15.90:0/2265227823 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40010720 con 0x7f6c50105ed0 2024-06-02T15:46:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.916+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c501a2d30 con 0x7f6c50105ed0 2024-06-02T15:46:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.917+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501a31d0 con 0x7f6c50105ed0 2024-06-02T15:46:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.921+0000 7f6c4d600640 1 -- 172.21.15.90:0/2265227823 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6c40005020 con 0x7f6c50105ed0 2024-06-02T15:46:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.923+0000 7f6c4d600640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 0x7f6c340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.923+0000 7f6c4d600640 1 -- 172.21.15.90:0/2265227823 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6c400e2d50 con 0x7f6c50105ed0 2024-06-02T15:46:56.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.924+0000 7f6c54a00640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 0x7f6c340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:56.925+0000 7f6c54a00640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 0x7f6c340b4eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c380095f0 tx=0x7f6c38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:57.427 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:57.428 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.435+0000 7f6c5a1bc740 1 Processor -- start 2024-06-02T15:46:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.435+0000 7f6c5a1bc740 1 -- start start 2024-06-02T15:46:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b4af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b52b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556447abab90 con 0x556447b4ab30 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556447aaa730 con 0x556447b50690 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556447aae770 con 0x556447b4b520 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b4af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41478/0 (socket says 172.21.15.90:41478) 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b4af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45190/0 (socket says 172.21.15.90:45190) 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.436+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2677507838 learned_addr learned my addr 172.21.15.90:0/2677507838 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c55400640 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b52b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2677507838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 msgr2=0x556447b52b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b52b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2677507838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 msgr2=0x556447b4af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b4af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.437+0000 7f6c4fe00640 1 -- 172.21.15.90:0/2677507838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556447a438b0 con 0x556447b4b520 2024-06-02T15:46:57.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.438+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c400a71d0 tx=0x7f6c400a9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f3a65db092457f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.438+0000 7f6c45c00640 1 -- 172.21.15.90:0/2677507838 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a9f60 con 0x556447b4b520 2024-06-02T15:46:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.438+0000 7f6c45c00640 1 -- 172.21.15.90:0/2677507838 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400ae310 con 0x556447b4b520 2024-06-02T15:46:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.438+0000 7f6c45c00640 1 -- 172.21.15.90:0/2677507838 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40005510 con 0x556447b4b520 2024-06-02T15:46:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/2677507838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 msgr2=0x556447b50020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c400a71d0 tx=0x7f6c400a9790 comp rx=0 tx=0).stop 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/2677507838 shutdown_connections 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b52b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b50020 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/2677507838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b4af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.439+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/2677507838 >> 172.21.15.90:0/2677507838 conn(0x556447b461a0 msgr2=0x556447b465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.440+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/2677507838 shutdown_connections 2024-06-02T15:46:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.440+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/2677507838 wait complete. 2024-06-02T15:46:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.505+0000 7f6c5a1bc740 1 -- start start 2024-06-02T15:46:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.511+0000 7f6c5a1bc740 1 Processor -- start 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b54990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556447790790 con 0x556447b4ab30 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556447a52bc0 con 0x556447b50690 2024-06-02T15:46:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556447b53640 con 0x556447b4b520 2024-06-02T15:46:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c55400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b54990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c55400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:40544/0 (socket says 172.21.15.90:40544) 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c55400640 1 -- 172.21.15.90:0/692519625 learned_addr learned my addr 172.21.15.90:0/692519625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c54a00640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b54990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45206/0 (socket says 172.21.15.90:45206) 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.512+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 -- 172.21.15.90:0/692519625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 msgr2=0x556447b54ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b54ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 -- 172.21.15.90:0/692519625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 msgr2=0x556447b54990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b54990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 -- 172.21.15.90:0/692519625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564477bf210 con 0x556447b50690 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c55400640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c4802a9f0 tx=0x7f6c48005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e81e0d10797fb12 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.513+0000 7f6c3e000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:46:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.514+0000 7f6c3e000640 1 -- 172.21.15.90:0/692519625 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48035040 con 0x556447b50690 2024-06-02T15:46:57.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.514+0000 7f6c3e000640 1 -- 172.21.15.90:0/692519625 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6c480048c0 con 0x556447b50690 2024-06-02T15:46:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.517+0000 7f6c5a1bc740 10 client.0 _ll_register_callbacks cb 0x556447a30400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.519+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c400ae310 con 0x556447b50690 2024-06-02T15:46:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.519+0000 7f6c3e000640 1 -- 172.21.15.90:0/692519625 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c48005d50 con 0x556447b50690 2024-06-02T15:46:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.519+0000 7f6c5a1bc740 10 client.25634 fetch_fsmap learned FSMap version 28 2024-06-02T15:46:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.520+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55644788b620 con 0x556447b50690 2024-06-02T15:46:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:57 smithi090 ceph-mon[31595]: pgmap v589: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c3e000640 1 -- 172.21.15.90:0/692519625 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c48004ac0 con 0x556447b50690 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 10 client.25634 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 20 client.25634 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447b60280 0x556447b62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 4 client.25634 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:46:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.648+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556447abcae0 con 0x556447b60280 2024-06-02T15:46:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.649+0000 7f6c54a00640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447b60280 0x556447b62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.650+0000 7f6c54a00640 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447b60280 0x556447b62740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a655b76af1a57dbd server_cookie=17901baa8dd0989b in_seq=0 out_seq=0 2024-06-02T15:46:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.650+0000 7f6c3e000640 10 client.25634 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.651+0000 7f6c3e000640 1 -- 172.21.15.90:0/692519625 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556447abcae0 con 0x556447b60280 2024-06-02T15:46:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.651+0000 7f6c3e000640 10 client.25634 handle_command_reply: tid=0 2024-06-02T15:46:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.651+0000 7f6c5a1bc740 1 client.25634 shutdown 2024-06-02T15:46:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 msgr2=0x556447b5bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c4802a9f0 tx=0x7f6c48005270 comp rx=0 tx=0).stop 2024-06-02T15:46:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 shutdown_connections 2024-06-02T15:46:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447b60280 0x556447b62740 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b50690 0x556447b5bf10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b4b520 0x556447b54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c3e000640 0 client.25634 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/692519625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b54990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.652+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 >> 172.21.15.90:0/692519625 conn(0x556447b461a0 msgr2=0x556447b45310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.653+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 shutdown_connections 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.653+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/692519625 wait complete. 2024-06-02T15:46:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.653+0000 7f6c5a1bc740 20 client.25634 trim_cache size 0 max 16384 2024-06-02T15:46:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.688+0000 7f6c5a1bc740 1 Processor -- start 2024-06-02T15:46:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.688+0000 7f6c5a1bc740 1 -- start start 2024-06-02T15:46:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.688+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b44fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501afff0 con 0x556447b4ab30 2024-06-02T15:46:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ad820 con 0x556447b4b520 2024-06-02T15:46:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ab670 con 0x556447b50690 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:40548/0 (socket says 172.21.15.90:40548) 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c4fe00640 1 -- 172.21.15.90:0/134969446 learned_addr learned my addr 172.21.15.90:0/134969446 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.689+0000 7f6c55400640 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b44fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 -- 172.21.15.90:0/134969446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 msgr2=0x556447b44fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b44fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 -- 172.21.15.90:0/134969446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 msgr2=0x556447b44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 -- 172.21.15.90:0/134969446 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556447b503c0 con 0x556447b4b520 2024-06-02T15:46:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c400a9ef0 tx=0x7f6c400058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cfaab74e6343743 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.690+0000 7f6c45200640 1 -- 172.21.15.90:0/134969446 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b2dc0 con 0x556447b4b520 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.691+0000 7f6c45200640 1 -- 172.21.15.90:0/134969446 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400a61f0 con 0x556447b4b520 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.691+0000 7f6c45200640 1 -- 172.21.15.90:0/134969446 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a58d0 con 0x556447b4b520 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.691+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/134969446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 msgr2=0x556447b5bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.691+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c400a9ef0 tx=0x7f6c400058a0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/134969446 shutdown_connections 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b44fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4b520 0x556447b5bed0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/134969446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4ab30 0x556447b44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/134969446 >> 172.21.15.90:0/134969446 conn(0x556447b461a0 msgr2=0x556447b49710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/134969446 shutdown_connections 2024-06-02T15:46:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/134969446 wait complete. 2024-06-02T15:46:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 -- start start 2024-06-02T15:46:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.692+0000 7f6c5a1bc740 1 Processor -- start 2024-06-02T15:46:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 0x556447b59790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 0x556447b59cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501abce0 con 0x556447b4b520 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ad150 con 0x556447b4ab30 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501ac3b0 con 0x556447b50690 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c5a1bc740 10 client.0 _ll_register_callbacks cb 0x7f6c400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c4fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 0x556447b59cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 0x556447b59790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c55400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c54a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 0x556447b59790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:40552/0 (socket says 172.21.15.90:40552) 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c55400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b5ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41502/0 (socket says 172.21.15.90:41502) 2024-06-02T15:46:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.693+0000 7f6c54a00640 1 -- 172.21.15.90:0/424521461 learned_addr learned my addr 172.21.15.90:0/424521461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 -- 172.21.15.90:0/424521461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 msgr2=0x556447b5ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b5ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 -- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 msgr2=0x556447b59790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 0x556447b59790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 -- 172.21.15.90:0/424521461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556447b5b1f0 con 0x556447b4b520 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c4fe00640 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 0x556447b59cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c760 tx=0x7f6c400a9b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7ae1773ce8d61a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c3e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a91c0 con 0x556447b4b520 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6c400a9360 con 0x556447b4b520 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.694+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c400a61f0 con 0x556447b4b520 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c40089050 con 0x556447b4b520 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-06-02T15:46:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556447bcafd0 con 0x556447b4b520 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400a61f0 con 0x556447b4b520 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6c400ba070 con 0x556447b4b520 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 20 client.15501 populate_metadata read hostname 'smithi090' 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447bcb1f0 0x556447bcd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 4 client.15501 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.695+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556447ab62c0 con 0x556447bcb1f0 2024-06-02T15:46:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.696+0000 7f6c54a00640 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447bcb1f0 0x556447bcd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:46:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.696+0000 7f6c54a00640 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447bcb1f0 0x556447bcd5e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e76fe6c32997ef4 server_cookie=c4c93c865c276b13 in_seq=0 out_seq=0 2024-06-02T15:46:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.696+0000 7f6c3e000640 10 client.15501 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.709+0000 7f6c3e000640 1 -- 172.21.15.90:0/424521461 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556447ab62c0 con 0x556447bcb1f0 2024-06-02T15:46:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.709+0000 7f6c3e000640 10 client.15501 handle_command_reply: tid=0 2024-06-02T15:46:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 client.15501 shutdown 2024-06-02T15:46:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 msgr2=0x556447b59cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 0x556447b59cd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c760 tx=0x7f6c400a9b30 comp rx=0 tx=0).stop 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 shutdown_connections 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556447bcb1f0 0x556447bcd5e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556447b50690 0x556447b5ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c3e000640 0 client.15501 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556447b4b520 0x556447b59cd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 --2- 172.21.15.90:0/424521461 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556447b4ab30 0x556447b59790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 >> 172.21.15.90:0/424521461 conn(0x556447b461a0 msgr2=0x556447b4bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 shutdown_connections 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.710+0000 7f6c5a1bc740 1 -- 172.21.15.90:0/424521461 wait complete. 2024-06-02T15:46:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.711+0000 7f6c5a1bc740 20 client.15501 trim_cache size 0 max 16384 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.711+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 msgr2=0x7f6c340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.711+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 0x7f6c340b4eb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c380095f0 tx=0x7f6c38005f90 comp rx=0 tx=0).stop 2024-06-02T15:46:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.711+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 msgr2=0x7f6c5019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.711+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009fd0 tx=0x7f6c4000ef90 comp rx=0 tx=0).stop 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 shutdown_connections 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f6c340b29f0 0x7f6c340b4eb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c50106890 0x7f6c501a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c50105ed0 0x7f6c5019b360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 --2- 172.21.15.90:0/2265227823 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6c50104ce0 0x7f6c5019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 >> 172.21.15.90:0/2265227823 conn(0x7f6c50100270 msgr2=0x7f6c50109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 shutdown_connections 2024-06-02T15:46:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:46:57.712+0000 7f6c57400640 1 -- 172.21.15.90:0/2265227823 wait complete. 2024-06-02T15:46:57.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:57 smithi120 ceph-mon[36451]: pgmap v589: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:46:57.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:57 smithi135 ceph-mon[36839]: pgmap v589: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:46:58.136 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:46:58.136 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:46:58.199 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:58.199 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:46:58.199 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 50 -lt 100 ']' 2024-06-02T15:46:58.200 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 50 + 1 2024-06-02T15:46:58.310 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=51 2024-06-02T15:46:58.310 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:46:58.390 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:46:58.390 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:46:58.394 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:46:58.396 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:46:58.396 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:46:59.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:46:59 smithi090 ceph-mon[31595]: pgmap v590: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-06-02T15:46:59.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:46:59 smithi120 ceph-mon[36451]: pgmap v590: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-06-02T15:46:59.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:46:59 smithi135 ceph-mon[36839]: pgmap v590: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 KiB/s wr, 8 op/s 2024-06-02T15:47:00.261 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:00.261 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:01.044 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:01.044 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:01.084 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:01.084 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:01.084 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 51 -lt 100 ']' 2024-06-02T15:47:01.086 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 51 + 1 2024-06-02T15:47:01.278 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=52 2024-06-02T15:47:01.278 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:01.366 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:01.366 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:01.371 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:01.372 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:01.373 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:01.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:01 smithi120 ceph-mon[36451]: pgmap v591: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-06-02T15:47:01.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:01 smithi135 ceph-mon[36839]: pgmap v591: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-06-02T15:47:01.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:01 smithi090 ceph-mon[31595]: pgmap v591: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.5 KiB/s wr, 9 op/s 2024-06-02T15:47:03.386 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:03.386 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:03 smithi120 ceph-mon[36451]: pgmap v592: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-02T15:47:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:03 smithi135 ceph-mon[36839]: pgmap v592: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-02T15:47:03.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:03 smithi090 ceph-mon[31595]: pgmap v592: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-06-02T15:47:03.927 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:03.927 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:03.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:03.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:03.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 52 -lt 100 ']' 2024-06-02T15:47:03.968 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 52 + 1 2024-06-02T15:47:04.097 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=53 2024-06-02T15:47:04.097 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:04.207 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:04.207 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:04.212 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:04.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:04.215 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:05.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:05 smithi090 ceph-mon[31595]: pgmap v593: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:05.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:05 smithi120 ceph-mon[36451]: pgmap v593: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:05.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:05 smithi135 ceph-mon[36839]: pgmap v593: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:06.063 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:06.063 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:06.503 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:06.503 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:06.541 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:06.541 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:06.541 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 53 -lt 100 ']' 2024-06-02T15:47:06.543 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 53 + 1 2024-06-02T15:47:06.607 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=54 2024-06-02T15:47:06.607 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:06.682 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:06.682 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:06.688 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:06.688 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:06.689 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:07.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:07 smithi120 ceph-mon[36451]: pgmap v594: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-02T15:47:07.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:07 smithi135 ceph-mon[36839]: pgmap v594: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-02T15:47:07.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:07 smithi090 ceph-mon[31595]: pgmap v594: 129 pgs: 129 active+clean; 1.5 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 12 op/s 2024-06-02T15:47:08.535 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:08.536 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:08.833 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:08.834 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:08.861 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:08.861 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:08.861 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 54 -lt 100 ']' 2024-06-02T15:47:08.863 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 54 + 1 2024-06-02T15:47:08.951 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=55 2024-06-02T15:47:08.951 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:09.143 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:09.144 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:09.148 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:09.149 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:09.150 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:09 smithi120 ceph-mon[36451]: pgmap v595: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:09 smithi135 ceph-mon[36839]: pgmap v595: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:09.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:09 smithi090 ceph-mon[31595]: pgmap v595: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:10.860 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:10.861 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:11.216 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:11.216 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:11.247 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:11.247 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:11.247 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 55 -lt 100 ']' 2024-06-02T15:47:11.249 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 55 + 1 2024-06-02T15:47:11.388 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=56 2024-06-02T15:47:11.389 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:11.484 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:11.484 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:11.488 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:11.489 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:11.490 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:11.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:11 smithi120 ceph-mon[36451]: pgmap v596: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-06-02T15:47:11.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:11 smithi135 ceph-mon[36839]: pgmap v596: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-06-02T15:47:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:11 smithi090 ceph-mon[31595]: pgmap v596: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-06-02T15:47:13.477 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:13.477 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:13 smithi120 ceph-mon[36451]: pgmap v597: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:47:13.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:13 smithi135 ceph-mon[36839]: pgmap v597: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:47:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:13 smithi090 ceph-mon[31595]: pgmap v597: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-06-02T15:47:13.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 -- 172.21.15.90:0/2173617627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 msgr2=0x7f604c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:13.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 --2- 172.21.15.90:0/2173617627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c113e90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038009ca0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 -- 172.21.15.90:0/2173617627 shutdown_connections 2024-06-02T15:47:13.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 --2- 172.21.15.90:0/2173617627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 --2- 172.21.15.90:0/2173617627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 --2- 172.21.15.90:0/2173617627 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c113e90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.962+0000 7f6050c00700 1 -- 172.21.15.90:0/2173617627 >> 172.21.15.90:0/2173617627 conn(0x7f604c101220 msgr2=0x7f604c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.963+0000 7f6050c00700 1 -- 172.21.15.90:0/2173617627 shutdown_connections 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.963+0000 7f6050c00700 1 -- 172.21.15.90:0/2173617627 wait complete. 2024-06-02T15:47:13.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.964+0000 7f6050c00700 1 Processor -- start 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.964+0000 7f6050c00700 1 -- start start 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f604c119d90 con 0x7f604c077a70 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c119c10 con 0x7f604c078390 2024-06-02T15:47:13.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6050c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f604c119f10 con 0x7f604c1143d0 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f604a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6049e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6049e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60178/0 (socket says 172.21.15.90:60178) 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6049e00700 1 -- 172.21.15.90:0/1864968347 learned_addr learned my addr 172.21.15.90:0/1864968347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f604a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35356/0 (socket says 172.21.15.90:35356) 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.965+0000 7f6049400700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 -- 172.21.15.90:0/1864968347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 msgr2=0x7f604c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 -- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 msgr2=0x7f604c1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:13.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 -- 172.21.15.90:0/1864968347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1a52b0 con 0x7f604c1143d0 2024-06-02T15:47:13.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.966+0000 7f604a800700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f604c07bd60 tx=0x7f604400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:13.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.967+0000 7f6042a00700 1 -- 172.21.15.90:0/1864968347 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440092e0 con 0x7f604c1143d0 2024-06-02T15:47:13.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.967+0000 7f6042a00700 1 -- 172.21.15.90:0/1864968347 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6044004bb0 con 0x7f604c1143d0 2024-06-02T15:47:13.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.967+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1a5540 con 0x7f604c1143d0 2024-06-02T15:47:13.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.967+0000 7f6042a00700 1 -- 172.21.15.90:0/1864968347 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044004020 con 0x7f604c1143d0 2024-06-02T15:47:13.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.967+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1ace20 con 0x7f604c1143d0 2024-06-02T15:47:13.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.970+0000 7f6042a00700 1 -- 172.21.15.90:0/1864968347 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f604401b030 con 0x7f604c1143d0 2024-06-02T15:47:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.972+0000 7f6042a00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 0x7f60300b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.972+0000 7f6042a00700 1 -- 172.21.15.90:0/1864968347 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f60440e1fa0 con 0x7f604c1143d0 2024-06-02T15:47:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.972+0000 7f6049e00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 0x7f60300b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:13.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:13.973+0000 7f6049e00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 0x7f60300b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f603c009d20 tx=0x7f603c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:14.009 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:14.010 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:14.040 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:14.041 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:14.041 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 56 -lt 100 ']' 2024-06-02T15:47:14.041 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 56 + 1 2024-06-02T15:47:14.176 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=57 2024-06-02T15:47:14.177 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:14.265 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:14.266 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:14.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:14.271 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:14.272 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:14.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.388+0000 7f605b5ffb80 1 Processor -- start 2024-06-02T15:47:14.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.389+0000 7f605b5ffb80 1 -- start start 2024-06-02T15:47:14.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.389+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10727eb0 0x564b10728290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.389+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 0x564b10722260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.389+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.389+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564b10680e40 con 0x564b107287d0 2024-06-02T15:47:14.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564b106709e0 con 0x564b10722880 2024-06-02T15:47:14.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b10674a20 con 0x564b10727eb0 2024-06-02T15:47:14.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f6049400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 0x564b10722260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f604a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f6049e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10727eb0 0x564b10728290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f604a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59414/0 (socket says 172.21.15.90:59414) 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.390+0000 7f604a800700 1 -- 172.21.15.90:0/3153056559 learned_addr learned my addr 172.21.15.90:0/3153056559 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 -- 172.21.15.90:0/3153056559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10727eb0 msgr2=0x564b10728290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10727eb0 0x564b10728290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 -- 172.21.15.90:0/3153056559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 msgr2=0x564b10722260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 0x564b10722260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 -- 172.21.15.90:0/3153056559 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b10639420 con 0x564b10722880 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f6049400700 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 0x564b10722260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f604a800700 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60440a8ff0 tx=0x7f60440a8b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38215e594cedcf3f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f6037400700 1 -- 172.21.15.90:0/3153056559 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440a92f0 con 0x564b10722880 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f6037400700 1 -- 172.21.15.90:0/3153056559 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60440ad5f0 con 0x564b10722880 2024-06-02T15:47:14.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.391+0000 7f6037400700 1 -- 172.21.15.90:0/3153056559 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440a8550 con 0x564b10722880 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3153056559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 msgr2=0x564b10722ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60440a8ff0 tx=0x7f60440a8b60 comp rx=0 tx=0).stop 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3153056559 shutdown_connections 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722ce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10727eb0 0x564b10728290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3153056559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b107287d0 0x564b10722260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3153056559 >> 172.21.15.90:0/3153056559 conn(0x564b10650be0 msgr2=0x564b1071d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3153056559 shutdown_connections 2024-06-02T15:47:14.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.393+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3153056559 wait complete. 2024-06-02T15:47:14.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.394+0000 7f605b5ffb80 1 -- start start 2024-06-02T15:47:14.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.394+0000 7f605b5ffb80 1 Processor -- start 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b1072e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1072b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1072bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564b10576fa0 con 0x564b10727eb0 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564b10611760 con 0x564b107287d0 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564b1072f240 con 0x564b10722880 2024-06-02T15:47:14.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f605b5ffb80 10 client.0 _ll_register_callbacks cb 0x564b1061ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:14.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.395+0000 7f6049400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1072b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1072bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f6049400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1072b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60194/0 (socket says 172.21.15.90:60194) 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f6049400700 1 -- 172.21.15.90:0/3974346386 learned_addr learned my addr 172.21.15.90:0/3974346386 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f6049e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b1072e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 -- 172.21.15.90:0/3974346386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 msgr2=0x564b1072e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b1072e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 -- 172.21.15.90:0/3974346386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 msgr2=0x564b1072b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1072b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.396+0000 7f604a800700 1 -- 172.21.15.90:0/3974346386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b105c08d0 con 0x564b107287d0 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f604a800700 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1072bdf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60440a9950 tx=0x7f60440b1db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517c27c2163d1e42 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:14.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f602be00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:47:14.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440dfd60 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f60440a8550 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60440ad5f0 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60440ca170 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f605b5ffb80 10 client.25652 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.397+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564b10641060 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60440ad5f0 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f60440a8150 con 0x564b107287d0 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 10 client.25652 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 20 client.25652 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b1073bc20 0x564b1073e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 4 client.25652 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:47:14.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.398+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b10682d90 con 0x564b1073bc20 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.399+0000 7f6049e00700 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b1073bc20 0x564b1073e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.399+0000 7f6049e00700 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b1073bc20 0x564b1073e010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21787551d0db43b0 server_cookie=b3ea72e3fa92a7a5 in_seq=0 out_seq=0 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.399+0000 7f602be00700 10 client.25652 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.400+0000 7f602be00700 1 -- 172.21.15.90:0/3974346386 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564b10682d90 con 0x564b1073bc20 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.400+0000 7f602be00700 10 client.25652 handle_command_reply: tid=0 2024-06-02T15:47:14.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.400+0000 7f605b5ffb80 1 client.25652 shutdown 2024-06-02T15:47:14.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 msgr2=0x564b1072bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1072bdf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60440a9950 tx=0x7f60440b1db0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 shutdown_connections 2024-06-02T15:47:14.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b1073bc20 0x564b1073e010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1072bdf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b1072e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f602be00700 0 client.25652 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3974346386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1072b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 >> 172.21.15.90:0/3974346386 conn(0x564b10650be0 msgr2=0x564b10721250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 shutdown_connections 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3974346386 wait complete. 2024-06-02T15:47:14.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.401+0000 7f605b5ffb80 20 client.25652 trim_cache size 0 max 16384 2024-06-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.449+0000 7f605b5ffb80 1 Processor -- start 2024-06-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.449+0000 7f605b5ffb80 1 -- start start 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1071d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b8300 con 0x564b10727eb0 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b5b60 con 0x564b10722880 2024-06-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b39b0 con 0x564b107287d0 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f6049e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f604a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f6049e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59436/0 (socket says 172.21.15.90:59436) 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f604a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35376/0 (socket says 172.21.15.90:35376) 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.450+0000 7f6049e00700 1 -- 172.21.15.90:0/422060746 learned_addr learned my addr 172.21.15.90:0/422060746 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049400700 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1071d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 -- 172.21.15.90:0/422060746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 msgr2=0x564b10728bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 -- 172.21.15.90:0/422060746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 msgr2=0x564b1071d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1071d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 -- 172.21.15.90:0/422060746 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b10727a70 con 0x564b10722880 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f604a800700 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.451+0000 7f6049e00700 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f603c011ab0 tx=0x7f603c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7470df876b07c73 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.452+0000 7f6036a00700 1 -- 172.21.15.90:0/422060746 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c038070 con 0x564b10722880 2024-06-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.452+0000 7f6036a00700 1 -- 172.21.15.90:0/422060746 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c033af0 con 0x564b10722880 2024-06-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.452+0000 7f6036a00700 1 -- 172.21.15.90:0/422060746 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c00eb40 con 0x564b10722880 2024-06-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 -- 172.21.15.90:0/422060746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 msgr2=0x564b10722260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f603c011ab0 tx=0x7f603c014000 comp rx=0 tx=0).stop 2024-06-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 -- 172.21.15.90:0/422060746 shutdown_connections 2024-06-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b107287d0 0x564b10728bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b10722880 0x564b10722260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/422060746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b1071d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.453+0000 7f605b5ffb80 1 -- 172.21.15.90:0/422060746 >> 172.21.15.90:0/422060746 conn(0x564b10650be0 msgr2=0x564b1071c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.454+0000 7f605b5ffb80 1 -- 172.21.15.90:0/422060746 shutdown_connections 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.454+0000 7f605b5ffb80 1 -- 172.21.15.90:0/422060746 wait complete. 2024-06-02T15:47:14.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.455+0000 7f605b5ffb80 1 -- start start 2024-06-02T15:47:14.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.455+0000 7f605b5ffb80 1 Processor -- start 2024-06-02T15:47:14.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b107414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1073f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b4020 con 0x564b10727eb0 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b5490 con 0x564b107287d0 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f604c1b46f0 con 0x564b10722880 2024-06-02T15:47:14.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f605b5ffb80 10 client.0 _ll_register_callbacks cb 0x564b1061ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:14.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f6049e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f604a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1073f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f6049e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35382/0 (socket says 172.21.15.90:35382) 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f6049e00700 1 -- 172.21.15.90:0/3126726398 learned_addr learned my addr 172.21.15.90:0/3126726398 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f6049400700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b107414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f604a800700 1 -- 172.21.15.90:0/3126726398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 msgr2=0x564b10740fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f604a800700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.456+0000 7f604a800700 1 -- 172.21.15.90:0/3126726398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 msgr2=0x564b107414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f604a800700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b107414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f604a800700 1 -- 172.21.15.90:0/3126726398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1073fc00 con 0x564b107287d0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f6049e00700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f604a800700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1073f4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60440a5370 tx=0x7f60440ad120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676be6111bece63e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f602be00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440ad5f0 con 0x564b107287d0 2024-06-02T15:47:14.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603c033af0 con 0x564b107287d0 2024-06-02T15:47:14.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f60440a8760 con 0x564b107287d0 2024-06-02T15:47:14.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.457+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60440a92f0 con 0x564b107287d0 2024-06-02T15:47:14.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 10 client.25658 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:14.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564b107a6ea0 con 0x564b107287d0 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c033af0 con 0x564b107287d0 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f60440a3a10 con 0x564b107287d0 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 10 client.25658 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 20 client.25658 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b107a7290 0x564b107a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 4 client.25658 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.458+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b1067c570 con 0x564b107a7290 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.459+0000 7f6049e00700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b107a7290 0x564b107a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f602be00700 10 client.25658 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f6049e00700 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b107a7290 0x564b107a96b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d6e35948758de2 server_cookie=b2b948f5f9e7bda0 in_seq=0 out_seq=0 2024-06-02T15:47:14.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f602be00700 1 -- 172.21.15.90:0/3126726398 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564b1067c570 con 0x564b107a7290 2024-06-02T15:47:14.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f602be00700 10 client.25658 handle_command_reply: tid=0 2024-06-02T15:47:14.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 client.25658 shutdown 2024-06-02T15:47:14.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 msgr2=0x564b1073f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1073f4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60440a5370 tx=0x7f60440ad120 comp rx=0 tx=0).stop 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 shutdown_connections 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564b107a7290 0x564b107a96b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564b107287d0 0x564b1073f4b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564b10722880 0x564b10740fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 --2- 172.21.15.90:0/3126726398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564b10727eb0 0x564b107414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f602be00700 0 client.25658 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.460+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 >> 172.21.15.90:0/3126726398 conn(0x564b10650be0 msgr2=0x564b1072a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.461+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 shutdown_connections 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.461+0000 7f605b5ffb80 1 -- 172.21.15.90:0/3126726398 wait complete. 2024-06-02T15:47:14.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.461+0000 7f605b5ffb80 20 client.25658 trim_cache size 0 max 16384 2024-06-02T15:47:14.461 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:47:14.461 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:47:14.461 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:47:14.461 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 msgr2=0x7f60300b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 0x7f60300b4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f603c009d20 tx=0x7f603c009260 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 msgr2=0x7f604c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f604c07bd60 tx=0x7f604400ef90 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 shutdown_connections 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60300b2900 0x7f60300b4dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f604c1143d0 0x7f604c1a4b70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f604c077a70 0x7f604c1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 --2- 172.21.15.90:0/1864968347 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f604c078390 0x7f604c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 >> 172.21.15.90:0/1864968347 conn(0x7f604c101220 msgr2=0x7f604c103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.462+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 shutdown_connections 2024-06-02T15:47:14.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:47:14.463+0000 7f6050c00700 1 -- 172.21.15.90:0/1864968347 wait complete. 2024-06-02T15:47:15.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:15 smithi090 ceph-mon[31595]: pgmap v598: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-02T15:47:15.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:15 smithi120 ceph-mon[36451]: pgmap v598: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-02T15:47:15.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:15 smithi135 ceph-mon[36839]: pgmap v598: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.7 KiB/s wr, 10 op/s 2024-06-02T15:47:16.303 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:16.303 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:16.784 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:16.784 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:16.839 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:16.839 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:16.839 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 57 -lt 100 ']' 2024-06-02T15:47:16.840 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 57 + 1 2024-06-02T15:47:16.939 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=58 2024-06-02T15:47:16.939 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:17.056 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:17.057 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:17.061 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:17.062 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:17.063 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:17.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:17 smithi120 ceph-mon[36451]: pgmap v599: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-06-02T15:47:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:17 smithi135 ceph-mon[36839]: pgmap v599: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-06-02T15:47:17.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:17 smithi090 ceph-mon[31595]: pgmap v599: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.5 KiB/s wr, 8 op/s 2024-06-02T15:47:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 -- 172.21.15.90:0/1247642811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 msgr2=0x7feb1c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 --2- 172.21.15.90:0/1247642811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c101100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009900 tx=0x7feb0c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 -- 172.21.15.90:0/1247642811 shutdown_connections 2024-06-02T15:47:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 --2- 172.21.15.90:0/1247642811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c102890 0x7feb1c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 --2- 172.21.15.90:0/1247642811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c101ed0 0x7feb1c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 --2- 172.21.15.90:0/1247642811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c101100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 -- 172.21.15.90:0/1247642811 >> 172.21.15.90:0/1247642811 conn(0x7feb1c0fc270 msgr2=0x7feb1c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 -- 172.21.15.90:0/1247642811 shutdown_connections 2024-06-02T15:47:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.650+0000 7feb23200640 1 -- 172.21.15.90:0/1247642811 wait complete. 2024-06-02T15:47:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.796+0000 7feb23200640 1 Processor -- start 2024-06-02T15:47:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.855+0000 7feb23200640 1 -- start start 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.856+0000 7feb23200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.856+0000 7feb23200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c101ed0 0x7feb1c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.856+0000 7feb23200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c102890 0x7feb1c1a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb23200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10c3d0 con 0x7feb1c102890 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb23200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10c250 con 0x7feb1c101ed0 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb23200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10c550 con 0x7feb1c100ce0 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb1be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb1be00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35390/0 (socket says 172.21.15.90:35390) 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb1be00640 1 -- 172.21.15.90:0/3446018999 learned_addr learned my addr 172.21.15.90:0/3446018999 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb13400640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c101ed0 0x7feb1c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.857+0000 7feb21200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c102890 0x7feb1c1a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 -- 172.21.15.90:0/3446018999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c101ed0 msgr2=0x7feb1c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c101ed0 0x7feb1c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 -- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c102890 msgr2=0x7feb1c1a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c102890 0x7feb1c1a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 -- 172.21.15.90:0/3446018999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c1a2a80 con 0x7feb1c100ce0 2024-06-02T15:47:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.858+0000 7feb1be00640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb0c006010 tx=0x7feb0c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.899+0000 7feb19600640 1 -- 172.21.15.90:0/3446018999 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c03f030 con 0x7feb1c100ce0 2024-06-02T15:47:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.899+0000 7feb19600640 1 -- 172.21.15.90:0/3446018999 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c031910 con 0x7feb1c100ce0 2024-06-02T15:47:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.899+0000 7feb19600640 1 -- 172.21.15.90:0/3446018999 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c035070 con 0x7feb1c100ce0 2024-06-02T15:47:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.899+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c1a2d10 con 0x7feb1c100ce0 2024-06-02T15:47:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.907+0000 7feb19600640 1 -- 172.21.15.90:0/3446018999 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7feb0c042030 con 0x7feb1c100ce0 2024-06-02T15:47:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.916+0000 7feb19600640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 0x7feb000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.917+0000 7feb13400640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 0x7feb000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.918+0000 7feb13400640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 0x7feb000b4eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb1c19c370 tx=0x7feb04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.928+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a31b0 con 0x7feb1c100ce0 2024-06-02T15:47:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:18.929+0000 7feb19600640 1 -- 172.21.15.90:0/3446018999 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb0c0c5bb0 con 0x7feb1c100ce0 2024-06-02T15:47:19.268 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:19.269 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.573+0000 7feb2619a740 1 Processor -- start 2024-06-02T15:47:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.574+0000 7feb2619a740 1 -- start start 2024-06-02T15:47:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.574+0000 7feb2619a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.574+0000 7feb2619a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89d520 0x55e79b8a2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb2619a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb2619a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e79b80cb90 con 0x55e79b89d520 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb2619a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e79b7fc730 con 0x55e79b89cb30 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb2619a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e79b800770 con 0x55e79b8a2690 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb13400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89d520 0x55e79b8a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59488/0 (socket says 172.21.15.90:59488) 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb1be00640 1 -- 172.21.15.90:0/3775047754 learned_addr learned my addr 172.21.15.90:0/3775047754 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.575+0000 7feb21200640 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 -- 172.21.15.90:0/3775047754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 msgr2=0x55e79b8a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 -- 172.21.15.90:0/3775047754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89d520 msgr2=0x55e79b8a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89d520 0x55e79b8a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 -- 172.21.15.90:0/3775047754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79b7958b0 con 0x55e79b89cb30 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb1be00640 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb0c032af0 tx=0x7feb0c0ff970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db2341bc62dc435 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb11600640 1 -- 172.21.15.90:0/3775047754 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0c95c0 con 0x55e79b89cb30 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb11600640 1 -- 172.21.15.90:0/3775047754 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c0cd970 con 0x55e79b89cb30 2024-06-02T15:47:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.576+0000 7feb11600640 1 -- 172.21.15.90:0/3775047754 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0d21f0 con 0x55e79b89cb30 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 -- 172.21.15.90:0/3775047754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 msgr2=0x55e79b89cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb0c032af0 tx=0x7feb0c0ff970 comp rx=0 tx=0).stop 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 -- 172.21.15.90:0/3775047754 shutdown_connections 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89d520 0x55e79b8a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 --2- 172.21.15.90:0/3775047754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b89cf50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 -- 172.21.15.90:0/3775047754 >> 172.21.15.90:0/3775047754 conn(0x55e79b8981a0 msgr2=0x55e79b8985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.577+0000 7feb2619a740 1 -- 172.21.15.90:0/3775047754 shutdown_connections 2024-06-02T15:47:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.578+0000 7feb2619a740 1 -- 172.21.15.90:0/3775047754 wait complete. 2024-06-02T15:47:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.615+0000 7feb2619a740 1 -- start start 2024-06-02T15:47:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.625+0000 7feb2619a740 1 Processor -- start 2024-06-02T15:47:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b8adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e79b4e2790 con 0x55e79b8a2690 2024-06-02T15:47:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e79b7a4bc0 con 0x55e79b89cb30 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.626+0000 7feb2619a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e79b7fc730 con 0x55e79b89d520 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb21200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b8adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb21200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b8adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60246/0 (socket says 172.21.15.90:60246) 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb21200640 1 -- 172.21.15.90:0/2442020861 learned_addr learned my addr 172.21.15.90:0/2442020861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb13400640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 -- 172.21.15.90:0/2442020861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 msgr2=0x55e79b8a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 -- 172.21.15.90:0/2442020861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 msgr2=0x55e79b8adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b8adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.627+0000 7feb1be00640 1 -- 172.21.15.90:0/2442020861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79b511210 con 0x55e79b89cb30 2024-06-02T15:47:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.628+0000 7feb1be00640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a65a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0c5640 tx=0x7feb0c005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c48dc3ed5da01a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.628+0000 7feb0a000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:47:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.628+0000 7feb0a000640 1 -- 172.21.15.90:0/2442020861 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0d6020 con 0x55e79b89cb30 2024-06-02T15:47:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.628+0000 7feb0a000640 1 -- 172.21.15.90:0/2442020861 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb0c0d2190 con 0x55e79b89cb30 2024-06-02T15:47:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.632+0000 7feb2619a740 10 client.0 _ll_register_callbacks cb 0x55e79b782400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.633+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c0cd970 con 0x55e79b89cb30 2024-06-02T15:47:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.634+0000 7feb0a000640 1 -- 172.21.15.90:0/2442020861 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb0c0d2420 con 0x55e79b89cb30 2024-06-02T15:47:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.634+0000 7feb2619a740 10 client.25670 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.634+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55e79b5dd620 con 0x55e79b89cb30 2024-06-02T15:47:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.727+0000 7feb0a000640 1 -- 172.21.15.90:0/2442020861 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feb0c0c3ce0 con 0x55e79b89cb30 2024-06-02T15:47:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.727+0000 7feb2619a740 10 client.25670 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.727+0000 7feb2619a740 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.728+0000 7feb2619a740 20 client.25670 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.728+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b8b1c20 0x55e79b8b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.728+0000 7feb2619a740 4 client.25670 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.728+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e79b80eae0 con 0x55e79b8b1c20 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.728+0000 7feb13400640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b8b1c20 0x55e79b8b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.729+0000 7feb13400640 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b8b1c20 0x55e79b8b40b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d207b7c8a9bb39 server_cookie=74539ede6a798762 in_seq=0 out_seq=0 2024-06-02T15:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.730+0000 7feb0a000640 10 client.25670 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.731+0000 7feb0a000640 1 -- 172.21.15.90:0/2442020861 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e79b80eae0 con 0x55e79b8b1c20 2024-06-02T15:47:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.731+0000 7feb0a000640 10 client.25670 handle_command_reply: tid=0 2024-06-02T15:47:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.731+0000 7feb2619a740 1 client.25670 shutdown 2024-06-02T15:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 msgr2=0x55e79b8a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a65a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0c5640 tx=0x7feb0c005b90 comp rx=0 tx=0).stop 2024-06-02T15:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 shutdown_connections 2024-06-02T15:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b8b1c20 0x55e79b8b40b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b8adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 --2- 172.21.15.90:0/2442020861 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a65a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb0a000640 0 client.25670 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.732+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 >> 172.21.15.90:0/2442020861 conn(0x55e79b8981a0 msgr2=0x55e79b897310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.733+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 shutdown_connections 2024-06-02T15:47:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.733+0000 7feb2619a740 1 -- 172.21.15.90:0/2442020861 wait complete. 2024-06-02T15:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.733+0000 7feb2619a740 20 client.25670 trim_cache size 0 max 16384 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 Processor -- start 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 -- start start 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b896a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1afff0 con 0x55e79b8a2690 2024-06-02T15:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ad820 con 0x55e79b89cb30 2024-06-02T15:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.770+0000 7feb2619a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ab670 con 0x55e79b89d520 2024-06-02T15:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb13400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb21200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b896a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb13400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35422/0 (socket says 172.21.15.90:35422) 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59498/0 (socket says 172.21.15.90:59498) 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb13400640 1 -- 172.21.15.90:0/464031457 learned_addr learned my addr 172.21.15.90:0/464031457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 -- 172.21.15.90:0/464031457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 msgr2=0x55e79b8adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 -- 172.21.15.90:0/464031457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 msgr2=0x55e79b896a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b896a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb1be00640 1 -- 172.21.15.90:0/464031457 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79b897fd0 con 0x55e79b89cb30 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.771+0000 7feb21200640 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b896a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.772+0000 7feb13400640 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.772+0000 7feb1be00640 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0d25e0 tx=0x7feb0c0e9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5abf6e8a30ea02d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.772+0000 7feb10c00640 1 -- 172.21.15.90:0/464031457 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0d6050 con 0x55e79b89cb30 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.772+0000 7feb10c00640 1 -- 172.21.15.90:0/464031457 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c0cd970 con 0x55e79b89cb30 2024-06-02T15:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.772+0000 7feb10c00640 1 -- 172.21.15.90:0/464031457 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0d21b0 con 0x55e79b89cb30 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 -- 172.21.15.90:0/464031457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 msgr2=0x55e79b8a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0d25e0 tx=0x7feb0c0e9000 comp rx=0 tx=0).stop 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 -- 172.21.15.90:0/464031457 shutdown_connections 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b8a2690 0x55e79b896a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b89d520 0x55e79b8adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 --2- 172.21.15.90:0/464031457 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89cb30 0x55e79b8a6820 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 -- 172.21.15.90:0/464031457 >> 172.21.15.90:0/464031457 conn(0x55e79b8981a0 msgr2=0x55e79b89b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 -- 172.21.15.90:0/464031457 shutdown_connections 2024-06-02T15:47:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.774+0000 7feb2619a740 1 -- 172.21.15.90:0/464031457 wait complete. 2024-06-02T15:47:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.775+0000 7feb2619a740 1 -- start start 2024-06-02T15:47:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.776+0000 7feb2619a740 1 Processor -- start 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 0x55e79b8ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 0x55e79b8ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1abce0 con 0x55e79b89cb30 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ad150 con 0x55e79b89d520 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ac3b0 con 0x55e79b8a2690 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb2619a740 10 client.0 _ll_register_callbacks cb 0x7feb04009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb21200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb1be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 0x55e79b8ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 0x55e79b8ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb21200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35426/0 (socket says 172.21.15.90:35426) 2024-06-02T15:47:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.777+0000 7feb21200640 1 -- 172.21.15.90:0/137235081 learned_addr learned my addr 172.21.15.90:0/137235081 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 -- 172.21.15.90:0/137235081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 msgr2=0x55e79b8ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 -- 172.21.15.90:0/137235081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 msgr2=0x55e79b8ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 0x55e79b8ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 -- 172.21.15.90:0/137235081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79b8acdc0 con 0x55e79b89d520 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb1be00640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 0x55e79b8ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb21200640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb13400640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 0x55e79b8ab830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e79b89df90 tx=0x7feb04013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac036297f4710592 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb0a000640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0401bbd0 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c0cd970 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.778+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7feb0401bd70 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb04039050 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 10 client.25676 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e79b91c9f0 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c0cd970 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feb04029610 con 0x55e79b89d520 2024-06-02T15:47:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 10 client.25676 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 20 client.25676 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b91d3f0 0x55e79b91f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 4 client.25676 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.779+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e79b8082c0 con 0x55e79b91d3f0 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.780+0000 7feb1be00640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b91d3f0 0x55e79b91f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.780+0000 7feb1be00640 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b91d3f0 0x55e79b91f810 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc5c3efce6ed391 server_cookie=945d7c8039be2260 in_seq=0 out_seq=0 2024-06-02T15:47:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.780+0000 7feb0a000640 10 client.25676 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb0a000640 1 -- 172.21.15.90:0/137235081 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e79b8082c0 con 0x55e79b91d3f0 2024-06-02T15:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb0a000640 10 client.25676 handle_command_reply: tid=0 2024-06-02T15:47:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb2619a740 1 client.25676 shutdown 2024-06-02T15:47:19.793 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:19.794 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 msgr2=0x55e79b8ab830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 0x55e79b8ab830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e79b89df90 tx=0x7feb04013040 comp rx=0 tx=0).stop 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 shutdown_connections 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.796+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e79b91d3f0 0x55e79b91f810 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e79b8a2690 0x55e79b8ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb0a000640 0 client.25676 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e79b89d520 0x55e79b8ab830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 --2- 172.21.15.90:0/137235081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e79b89cb30 0x55e79b8ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 >> 172.21.15.90:0/137235081 conn(0x55e79b8981a0 msgr2=0x55e79b89b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 shutdown_connections 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 1 -- 172.21.15.90:0/137235081 wait complete. 2024-06-02T15:47:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.797+0000 7feb2619a740 20 client.25676 trim_cache size 0 max 16384 2024-06-02T15:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:47:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 msgr2=0x7feb000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 0x7feb000b4eb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb1c19c370 tx=0x7feb04005f90 comp rx=0 tx=0).stop 2024-06-02T15:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 msgr2=0x7feb1c19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb0c006010 tx=0x7feb0c0042f0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 shutdown_connections 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feb000b29f0 0x7feb000b4eb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb1c102890 0x7feb1c1a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feb1c101ed0 0x7feb1c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 --2- 172.21.15.90:0/3446018999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb1c100ce0 0x7feb1c19ae60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 >> 172.21.15.90:0/3446018999 conn(0x7feb1c0fc270 msgr2=0x7feb1c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 shutdown_connections 2024-06-02T15:47:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:19.798+0000 7feb23200640 1 -- 172.21.15.90:0/3446018999 wait complete. 2024-06-02T15:47:19.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:19 smithi120 ceph-mon[36451]: pgmap v600: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:47:19.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:19 smithi135 ceph-mon[36839]: pgmap v600: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:47:19.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:19.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:19.832 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 58 -lt 100 ']' 2024-06-02T15:47:19.833 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 58 + 1 2024-06-02T15:47:19.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:19 smithi090 ceph-mon[31595]: pgmap v600: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:47:19.904 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=59 2024-06-02T15:47:19.904 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:20.025 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:20.026 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:20.030 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:20.031 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:20.031 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:20.611 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:47:20.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:47:21.534 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:21.535 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:21 smithi090 ceph-mon[31595]: pgmap v601: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-06-02T15:47:21.777 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:21.777 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:21.802 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:21.802 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:21.802 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 59 -lt 100 ']' 2024-06-02T15:47:21.803 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 59 + 1 2024-06-02T15:47:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:21 smithi120 ceph-mon[36451]: pgmap v601: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-06-02T15:47:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:21 smithi135 ceph-mon[36839]: pgmap v601: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-06-02T15:47:21.866 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=60 2024-06-02T15:47:21.866 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:21.934 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:21.934 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:21.940 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:21.941 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:21.941 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:22.835 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:22.836 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:23.427 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:23.427 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:23.474 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:23.475 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:23.475 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 60 -lt 100 ']' 2024-06-02T15:47:23.475 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 60 + 1 2024-06-02T15:47:23.598 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=61 2024-06-02T15:47:23.598 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:23.784 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:23.785 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:23.790 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:23.790 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:23.791 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:23 smithi120 ceph-mon[36451]: pgmap v602: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-06-02T15:47:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:23 smithi135 ceph-mon[36839]: pgmap v602: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-06-02T15:47:23.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:23 smithi090 ceph-mon[31595]: pgmap v602: 129 pgs: 129 active+clean; 1.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-06-02T15:47:25.368 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:25.368 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:25.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:25 smithi120 ceph-mon[36451]: pgmap v603: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:25.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:25 smithi135 ceph-mon[36839]: pgmap v603: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:25.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:25 smithi090 ceph-mon[31595]: pgmap v603: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:47:25.901 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:25.901 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:25.938 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:25.938 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:25.938 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 61 -lt 100 ']' 2024-06-02T15:47:25.939 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 61 + 1 2024-06-02T15:47:26.071 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=62 2024-06-02T15:47:26.071 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:26.192 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:26.192 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:26.196 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:26.197 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:26.198 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:27 smithi090 ceph-mon[31595]: pgmap v604: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-02T15:47:27.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:27 smithi120 ceph-mon[36451]: pgmap v604: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-02T15:47:27.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:27 smithi135 ceph-mon[36839]: pgmap v604: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-06-02T15:47:27.951 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:27.951 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:28.543 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:28.543 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:28.582 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:28.582 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:28.582 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 62 -lt 100 ']' 2024-06-02T15:47:28.583 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 62 + 1 2024-06-02T15:47:28.887 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=63 2024-06-02T15:47:28.888 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:29.051 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:29.051 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:29.056 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:29.057 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:29.058 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:29.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:29 smithi120 ceph-mon[36451]: pgmap v605: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:47:29.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:29 smithi135 ceph-mon[36839]: pgmap v605: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:47:29.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:29 smithi090 ceph-mon[31595]: pgmap v605: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:47:30.901 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:30.902 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:31.451 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:31.451 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:31.516 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:31.516 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:31.516 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 63 -lt 100 ']' 2024-06-02T15:47:31.517 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 63 + 1 2024-06-02T15:47:31.584 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=64 2024-06-02T15:47:31.584 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:31.651 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:31.651 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:31.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:31.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:31.656 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:31.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:31 smithi120 ceph-mon[36451]: pgmap v606: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2024-06-02T15:47:31.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:31 smithi135 ceph-mon[36839]: pgmap v606: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2024-06-02T15:47:31.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:31 smithi090 ceph-mon[31595]: pgmap v606: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2024-06-02T15:47:33.117 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:33.118 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:33.601 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:33.601 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:33.641 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:33.642 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:33.643 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 64 -lt 100 ']' 2024-06-02T15:47:33.644 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 64 + 1 2024-06-02T15:47:33.729 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=65 2024-06-02T15:47:33.729 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:33.789 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:33.789 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:33.794 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:33.795 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:33.796 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:33.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:33 smithi120 ceph-mon[36451]: pgmap v607: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-06-02T15:47:33.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:33 smithi135 ceph-mon[36839]: pgmap v607: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-06-02T15:47:33.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:33 smithi090 ceph-mon[31595]: pgmap v607: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-06-02T15:47:35.542 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:35.543 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:35 smithi090 ceph-mon[31595]: pgmap v608: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:47:35.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:35 smithi120 ceph-mon[36451]: pgmap v608: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:47:35.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:35 smithi135 ceph-mon[36839]: pgmap v608: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-06-02T15:47:36.034 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:36.034 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:36.054 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:36.055 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:36.055 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 65 -lt 100 ']' 2024-06-02T15:47:36.056 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 65 + 1 2024-06-02T15:47:36.154 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=66 2024-06-02T15:47:36.154 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:36.213 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:36.213 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:36.218 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:36.219 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:36.219 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:37.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:37 smithi090 ceph-mon[31595]: pgmap v609: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-06-02T15:47:37.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:37 smithi120 ceph-mon[36451]: pgmap v609: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-06-02T15:47:37.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:37 smithi135 ceph-mon[36839]: pgmap v609: 129 pgs: 129 active+clean; 1.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-06-02T15:47:37.825 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:37.826 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:38.417 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:38.417 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:38.439 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:38.439 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:38.439 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 66 -lt 100 ']' 2024-06-02T15:47:38.440 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 66 + 1 2024-06-02T15:47:38.529 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=67 2024-06-02T15:47:38.529 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:38.638 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:38.639 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:38.644 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:38.645 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:38.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.645+0000 7f6636400640 1 -- 172.21.15.90:0/535145629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630101ed0 msgr2=0x7f6630102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.645+0000 7f6636400640 1 --2- 172.21.15.90:0/535145629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630101ed0 0x7f6630102350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f661c009860 tx=0x7f661c02a330 comp rx=0 tx=0).stop 2024-06-02T15:47:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 -- 172.21.15.90:0/535145629 shutdown_connections 2024-06-02T15:47:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 --2- 172.21.15.90:0/535145629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630102890 0x7f6630109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 --2- 172.21.15.90:0/535145629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630101ed0 0x7f6630102350 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 --2- 172.21.15.90:0/535145629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f6630101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 -- 172.21.15.90:0/535145629 >> 172.21.15.90:0/535145629 conn(0x7f66300fc270 msgr2=0x7f66300fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 -- 172.21.15.90:0/535145629 shutdown_connections 2024-06-02T15:47:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.646+0000 7f6636400640 1 -- 172.21.15.90:0/535145629 wait complete. 2024-06-02T15:47:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.697+0000 7f6636400640 1 Processor -- start 2024-06-02T15:47:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.697+0000 7f6636400640 1 -- start start 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630101ed0 0x7f663019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 0x7f66301a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f663010c380 con 0x7f6630102890 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f663010c200 con 0x7f6630100ce0 2024-06-02T15:47:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.698+0000 7f6636400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f663010c500 con 0x7f6630101ed0 2024-06-02T15:47:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53996/0 (socket says 172.21.15.90:53996) 2024-06-02T15:47:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 -- 172.21.15.90:0/4054828773 learned_addr learned my addr 172.21.15.90:0/4054828773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662ea00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630101ed0 0x7f663019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662fe00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 0x7f66301a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 -- 172.21.15.90:0/4054828773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630101ed0 msgr2=0x7f663019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630101ed0 0x7f663019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 -- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 msgr2=0x7f66301a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 0x7f66301a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 -- 172.21.15.90:0/4054828773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66301a29c0 con 0x7f6630100ce0 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662fe00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 0x7f66301a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:47:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.699+0000 7f662f400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f662000e9c0 tx=0x7f662000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.713+0000 7f6627e00640 1 -- 172.21.15.90:0/4054828773 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662000cde0 con 0x7f6630100ce0 2024-06-02T15:47:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.713+0000 7f6627e00640 1 -- 172.21.15.90:0/4054828773 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66200108e0 con 0x7f6630100ce0 2024-06-02T15:47:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.713+0000 7f6627e00640 1 -- 172.21.15.90:0/4054828773 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620018960 con 0x7f6630100ce0 2024-06-02T15:47:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.713+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66301a2cb0 con 0x7f6630100ce0 2024-06-02T15:47:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.719+0000 7f6627e00640 1 -- 172.21.15.90:0/4054828773 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f662001f020 con 0x7f6630100ce0 2024-06-02T15:47:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.721+0000 7f6627e00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 0x7f66140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.722+0000 7f662ea00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 0x7f66140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.722+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66301a31b0 con 0x7f6630100ce0 2024-06-02T15:47:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.723+0000 7f6627e00640 1 -- 172.21.15.90:0/4054828773 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f66200a5b60 con 0x7f6630100ce0 2024-06-02T15:47:39.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:39.723+0000 7f662ea00640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 0x7f66140b4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f661c02a920 tx=0x7f661c0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:39.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:39 smithi090 ceph-mon[31595]: pgmap v610: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-06-02T15:47:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:39 smithi120 ceph-mon[36451]: pgmap v610: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-06-02T15:47:39.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:39 smithi135 ceph-mon[36839]: pgmap v610: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 8 op/s 2024-06-02T15:47:39.830 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:47:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.075+0000 7f6639212740 1 Processor -- start 2024-06-02T15:47:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.076+0000 7f6639212740 1 -- start start 2024-06-02T15:47:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.076+0000 7f6639212740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.076+0000 7f6639212740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf24020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.076+0000 7f6639212740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf26b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f6639212740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d5aae8eb90 con 0x55d5aaf1eb30 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f6639212740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5aae7e730 con 0x55d5aaf24690 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f6639212740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d5aae82770 con 0x55d5aaf1f520 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51458/0 (socket says 172.21.15.90:51458) 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 -- 172.21.15.90:0/1424386620 learned_addr learned my addr 172.21.15.90:0/1424386620 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662ea00640 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf24020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662fe00640 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf26b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 -- 172.21.15.90:0/1424386620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 msgr2=0x55d5aaf24020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf24020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 -- 172.21.15.90:0/1424386620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 msgr2=0x55d5aaf26b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf26b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.077+0000 7f662f400640 1 -- 172.21.15.90:0/1424386620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5aae178b0 con 0x55d5aaf1eb30 2024-06-02T15:47:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.078+0000 7f662f400640 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f66200a9590 tx=0x7f66200df9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c939085c5a8c53 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.078+0000 7f6624c00640 1 -- 172.21.15.90:0/1424386620 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66200b2460 con 0x55d5aaf1eb30 2024-06-02T15:47:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.078+0000 7f6624c00640 1 -- 172.21.15.90:0/1424386620 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6620010a90 con 0x55d5aaf1eb30 2024-06-02T15:47:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.078+0000 7f6624c00640 1 -- 172.21.15.90:0/1424386620 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66200a8ac0 con 0x55d5aaf1eb30 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 -- 172.21.15.90:0/1424386620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 msgr2=0x55d5aaf1ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f66200a9590 tx=0x7f66200df9b0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 -- 172.21.15.90:0/1424386620 shutdown_connections 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf26b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 --2- 172.21.15.90:0/1424386620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf1ef50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.079+0000 7f6639212740 1 -- 172.21.15.90:0/1424386620 >> 172.21.15.90:0/1424386620 conn(0x55d5aaf1a1a0 msgr2=0x55d5aaf1a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.080+0000 7f6639212740 1 -- 172.21.15.90:0/1424386620 shutdown_connections 2024-06-02T15:47:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.080+0000 7f6639212740 1 -- 172.21.15.90:0/1424386620 wait complete. 2024-06-02T15:47:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.144+0000 7f6639212740 1 -- start start 2024-06-02T15:47:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.166+0000 7f6639212740 1 Processor -- start 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf31730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf38770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d5aab64790 con 0x55d5aaf1f520 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55d5aae26bc0 con 0x55d5aaf24690 2024-06-02T15:47:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f6639212740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d5aaf27640 con 0x55d5aaf1eb30 2024-06-02T15:47:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f662ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf31730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f662f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.167+0000 7f662ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf31730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51468/0 (socket says 172.21.15.90:51468) 2024-06-02T15:47:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52870/0 (socket says 172.21.15.90:52870) 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662ea00640 1 -- 172.21.15.90:0/814628898 learned_addr learned my addr 172.21.15.90:0/814628898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf38770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 -- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 msgr2=0x55d5aaf38770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf38770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 -- 172.21.15.90:0/814628898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 msgr2=0x55d5aaf31730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf31730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 -- 172.21.15.90:0/814628898 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5aab93210 con 0x55d5aaf1eb30 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f662f400640 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66200a67e0 tx=0x7f66200a7830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae65566444ba491 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.168+0000 7f6619600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.169+0000 7f6619600640 1 -- 172.21.15.90:0/814628898 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66200b6020 con 0x55d5aaf1eb30 2024-06-02T15:47:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.169+0000 7f6619600640 1 -- 172.21.15.90:0/814628898 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f66200ca580 con 0x55d5aaf1eb30 2024-06-02T15:47:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.170+0000 7f6639212740 10 client.0 _ll_register_callbacks cb 0x55d5aae04400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.172+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6620010a90 con 0x55d5aaf1eb30 2024-06-02T15:47:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.173+0000 7f6619600640 1 -- 172.21.15.90:0/814628898 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66200b2460 con 0x55d5aaf1eb30 2024-06-02T15:47:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.173+0000 7f6639212740 10 client.25651 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.173+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d5aac5f620 con 0x55d5aaf1eb30 2024-06-02T15:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6619600640 1 -- 172.21.15.90:0/814628898 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6620010430 con 0x55d5aaf1eb30 2024-06-02T15:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 10 client.25651 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 20 client.25651 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aaf3c740 0x55d5aaf3eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 4 client.25651 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:47:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.324+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5aae90ae0 con 0x55d5aaf3c740 2024-06-02T15:47:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.325+0000 7f662ea00640 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aaf3c740 0x55d5aaf3eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.326+0000 7f662ea00640 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aaf3c740 0x55d5aaf3eb30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f734c8a3f7f1bb1d server_cookie=55e3f48c7a0ab9d in_seq=0 out_seq=0 2024-06-02T15:47:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.326+0000 7f6619600640 10 client.25651 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.327+0000 7f6619600640 1 -- 172.21.15.90:0/814628898 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d5aae90ae0 con 0x55d5aaf3c740 2024-06-02T15:47:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.327+0000 7f6619600640 10 client.25651 handle_command_reply: tid=0 2024-06-02T15:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.328+0000 7f6639212740 1 client.25651 shutdown 2024-06-02T15:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 msgr2=0x55d5aaf311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66200a67e0 tx=0x7f66200a7830 comp rx=0 tx=0).stop 2024-06-02T15:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 shutdown_connections 2024-06-02T15:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aaf3c740 0x55d5aaf3eb30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aaf38770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6619600640 0 client.25651 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf31730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 --2- 172.21.15.90:0/814628898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf311f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 >> 172.21.15.90:0/814628898 conn(0x55d5aaf1a1a0 msgr2=0x55d5aaf19310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 shutdown_connections 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.329+0000 7f6639212740 1 -- 172.21.15.90:0/814628898 wait complete. 2024-06-02T15:47:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.330+0000 7f6639212740 20 client.25651 trim_cache size 0 max 16384 2024-06-02T15:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.376+0000 7f6639212740 1 Processor -- start 2024-06-02T15:47:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.376+0000 7f6639212740 1 -- start start 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf38730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf24690 0x55d5aaf1d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66301abcc0 con 0x55d5aaf1f520 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66301a94f0 con 0x55d5aaf1eb30 2024-06-02T15:47:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.377+0000 7f6639212740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66301a7340 con 0x55d5aaf24690 2024-06-02T15:47:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:54034/0 (socket says 172.21.15.90:54034) 2024-06-02T15:47:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 -- 172.21.15.90:0/3652905773 learned_addr learned my addr 172.21.15.90:0/3652905773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662fe00640 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf24690 0x55d5aaf1d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662ea00640 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf38730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 -- 172.21.15.90:0/3652905773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf24690 msgr2=0x55d5aaf1d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf24690 0x55d5aaf1d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 -- 172.21.15.90:0/3652905773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 msgr2=0x55d5aaf38730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf38730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.378+0000 7f662f400640 1 -- 172.21.15.90:0/3652905773 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5aaf1e770 con 0x55d5aaf1eb30 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.379+0000 7f662f400640 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6620018460 tx=0x7f66200c8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea43df4672f6dda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.379+0000 7f661be00640 1 -- 172.21.15.90:0/3652905773 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66200b6030 con 0x55d5aaf1eb30 2024-06-02T15:47:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.379+0000 7f661be00640 1 -- 172.21.15.90:0/3652905773 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6620010a90 con 0x55d5aaf1eb30 2024-06-02T15:47:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.379+0000 7f661be00640 1 -- 172.21.15.90:0/3652905773 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66200a8ac0 con 0x55d5aaf1eb30 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.380+0000 7f6639212740 1 -- 172.21.15.90:0/3652905773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 msgr2=0x55d5aaf31750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.380+0000 7f6639212740 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6620018460 tx=0x7f66200c8000 comp rx=0 tx=0).stop 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 -- 172.21.15.90:0/3652905773 shutdown_connections 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf24690 0x55d5aaf1d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1f520 0x55d5aaf38730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 --2- 172.21.15.90:0/3652905773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf31750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 -- 172.21.15.90:0/3652905773 >> 172.21.15.90:0/3652905773 conn(0x55d5aaf1a1a0 msgr2=0x55d5aaf1d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 -- 172.21.15.90:0/3652905773 shutdown_connections 2024-06-02T15:47:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.381+0000 7f6639212740 1 -- 172.21.15.90:0/3652905773 wait complete. 2024-06-02T15:47:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.382+0000 7f6639212740 1 -- start start 2024-06-02T15:47:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.382+0000 7f6639212740 1 Processor -- start 2024-06-02T15:47:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.383+0000 7f6639212740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.383+0000 7f6639212740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf35970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f6639212740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aafa5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f6639212740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f66301a79b0 con 0x55d5aaf1eb30 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f6639212740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66301a8e20 con 0x55d5aaf24690 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f6639212740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66301a8080 con 0x55d5aaf1f520 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f6639212740 10 client.0 _ll_register_callbacks cb 0x7f6620005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:47:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51488/0 (socket says 172.21.15.90:51488) 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662f400640 1 -- 172.21.15.90:0/4294341717 learned_addr learned my addr 172.21.15.90:0/4294341717 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aafa5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662ea00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf35970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 -- 172.21.15.90:0/4294341717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 msgr2=0x55d5aaf35970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf35970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 -- 172.21.15.90:0/4294341717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 msgr2=0x55d5aaf35430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662fe00640 1 -- 172.21.15.90:0/4294341717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5aaf27780 con 0x55d5aaf24690 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662f400640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.384+0000 7f662ea00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f662fe00640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aafa5130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55d5aaf15ad0 tx=0x7f6628018700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4566f2713dd33573 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f6619600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:47:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662801e070 con 0x55d5aaf24690 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6620010a90 con 0x55d5aaf24690 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f6628004b20 con 0x55d5aaf24690 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.385+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f662802a050 con 0x55d5aaf24690 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 10 client.25694 fetch_fsmap learned FSMap version 28 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5aafa6500 con 0x55d5aaf24690 2024-06-02T15:47:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6620010a90 con 0x55d5aaf24690 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66280045f0 con 0x55d5aaf24690 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 10 client.25694 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 20 client.25694 populate_metadata read hostname 'smithi090' 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aafa6f60 0x55d5aafa9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 4 client.25694 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.386+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5aae8a2c0 con 0x55d5aafa6f60 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.387+0000 7f662f400640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aafa6f60 0x55d5aafa9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.387+0000 7f662f400640 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aafa6f60 0x55d5aafa9380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9450983e1d6177 server_cookie=80a61451ac69e390 in_seq=0 out_seq=0 2024-06-02T15:47:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.387+0000 7f6619600640 10 client.25694 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:40.390 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:40.390 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6619600640 1 -- 172.21.15.90:0/4294341717 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d5aae8a2c0 con 0x55d5aafa6f60 2024-06-02T15:47:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6619600640 10 client.25694 handle_command_reply: tid=0 2024-06-02T15:47:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 client.25694 shutdown 2024-06-02T15:47:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 msgr2=0x55d5aafa5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aafa5130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55d5aaf15ad0 tx=0x7f6628018700 comp rx=0 tx=0).stop 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 shutdown_connections 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55d5aafa6f60 0x55d5aafa9380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55d5aaf24690 0x55d5aafa5130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6619600640 0 client.25694 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d5aaf1f520 0x55d5aaf35970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 --2- 172.21.15.90:0/4294341717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5aaf1eb30 0x55d5aaf35430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.410+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 >> 172.21.15.90:0/4294341717 conn(0x55d5aaf1a1a0 msgr2=0x55d5aaf1d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.411+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 shutdown_connections 2024-06-02T15:47:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.411+0000 7f6639212740 1 -- 172.21.15.90:0/4294341717 wait complete. 2024-06-02T15:47:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.411+0000 7f6639212740 20 client.25694 trim_cache size 0 max 16384 2024-06-02T15:47:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:47:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:47:40.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 msgr2=0x7f66140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 0x7f66140b4eb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f661c02a920 tx=0x7f661c0058b0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 msgr2=0x7f663019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f662000e9c0 tx=0x7f662000ee80 comp rx=0 tx=0).stop 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 shutdown_connections 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66140b29f0 0x7f66140b4eb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6630102890 0x7f66301a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6630101ed0 0x7f663019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 --2- 172.21.15.90:0/4054828773 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6630100ce0 0x7f663019adb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:47:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 >> 172.21.15.90:0/4054828773 conn(0x7f66300fc270 msgr2=0x7f6630105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:47:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 shutdown_connections 2024-06-02T15:47:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:47:40.413+0000 7f6636400640 1 -- 172.21.15.90:0/4054828773 wait complete. 2024-06-02T15:47:40.843 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:40.843 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:40.876 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:40.876 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:40.876 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 67 -lt 100 ']' 2024-06-02T15:47:40.878 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 67 + 1 2024-06-02T15:47:40.980 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=68 2024-06-02T15:47:40.980 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:41.023 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:41.023 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:41.027 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:41.028 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:41.029 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:41 smithi120 ceph-mon[36451]: pgmap v611: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:47:41.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:41 smithi135 ceph-mon[36839]: pgmap v611: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:47:41.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:41 smithi090 ceph-mon[31595]: pgmap v611: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-06-02T15:47:42.384 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:42.384 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:42.884 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:42.884 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:42.929 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:42.929 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:42.929 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 68 -lt 100 ']' 2024-06-02T15:47:42.931 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 68 + 1 2024-06-02T15:47:43.020 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=69 2024-06-02T15:47:43.020 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:43.097 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:43.097 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:43.102 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:43.104 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:43.104 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:43 smithi120 ceph-mon[36451]: pgmap v612: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:47:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:43 smithi135 ceph-mon[36839]: pgmap v612: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:47:43.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:43 smithi090 ceph-mon[31595]: pgmap v612: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.9 KiB/s wr, 10 op/s 2024-06-02T15:47:44.592 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:44.593 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:45.066 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:45.066 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:45.144 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:45.144 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:45.144 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 69 -lt 100 ']' 2024-06-02T15:47:45.146 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 69 + 1 2024-06-02T15:47:45.237 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=70 2024-06-02T15:47:45.237 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:45.331 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:45.332 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:45.336 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:45.338 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:45.339 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:45 smithi120 ceph-mon[36451]: pgmap v613: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:45.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:45 smithi135 ceph-mon[36839]: pgmap v613: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:45.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:45 smithi090 ceph-mon[31595]: pgmap v613: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:47:46.542 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:46.543 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:46 smithi120 ceph-mon[36451]: pgmap v614: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-02T15:47:46.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:46 smithi135 ceph-mon[36839]: pgmap v614: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-02T15:47:46.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:46 smithi090 ceph-mon[31595]: pgmap v614: 129 pgs: 129 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 12 op/s 2024-06-02T15:47:47.083 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:47.084 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:47.117 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:47.117 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:47.117 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 70 -lt 100 ']' 2024-06-02T15:47:47.118 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 70 + 1 2024-06-02T15:47:47.182 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=71 2024-06-02T15:47:47.182 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:47.246 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:47.246 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:47.251 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:47.251 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:47.252 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:48.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:48 smithi090 ceph-mon[31595]: pgmap v615: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:47:48.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:48 smithi120 ceph-mon[36451]: pgmap v615: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:47:48.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:48 smithi135 ceph-mon[36839]: pgmap v615: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-06-02T15:47:48.575 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:48.576 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:49.234 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:49.234 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:49.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:49.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:49.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 71 -lt 100 ']' 2024-06-02T15:47:49.336 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 71 + 1 2024-06-02T15:47:49.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:49 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:47:49.445 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=72 2024-06-02T15:47:49.445 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:49.525 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:49.525 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:49.529 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:49.530 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:49.530 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:49 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:47:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:49 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: Unable to set osd_memory_target on smithi135 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:47:50.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:50 smithi090 ceph-mon[31595]: pgmap v616: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: Unable to set osd_memory_target on smithi135 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:47:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:50 smithi120 ceph-mon[36451]: pgmap v616: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-02T15:47:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:47:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-06-02T15:47:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: Unable to set osd_memory_target on smithi135 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:47:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:50 smithi135 ceph-mon[36839]: pgmap v616: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.9 KiB/s wr, 13 op/s 2024-06-02T15:47:51.234 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:51.234 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:51.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:51.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:51.723 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:51.723 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:51.723 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 72 -lt 100 ']' 2024-06-02T15:47:51.724 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 72 + 1 2024-06-02T15:47:51.886 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=73 2024-06-02T15:47:51.887 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:52.059 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:52.059 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:52.064 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:52.065 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:52.066 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:52.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:52 smithi090 ceph-mon[31595]: pgmap v617: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:47:52.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:52 smithi120 ceph-mon[36451]: pgmap v617: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:47:52.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:52 smithi135 ceph-mon[36839]: pgmap v617: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-06-02T15:47:53.751 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:53.751 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:54.100 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:54.253 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 73 -lt 100 ']' 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 73 + 1 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=74 2024-06-02T15:47:54.254 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:54.283 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:54.284 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:54.288 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:54.289 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:54.290 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:54.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:54 smithi120 ceph-mon[36451]: pgmap v618: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-02T15:47:54.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:54 smithi135 ceph-mon[36839]: pgmap v618: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-02T15:47:54.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:54 smithi090 ceph-mon[31595]: pgmap v618: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-06-02T15:47:56.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:56 smithi090 ceph-mon[31595]: pgmap v619: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:47:56.475 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:56.475 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:56.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:56 smithi120 ceph-mon[36451]: pgmap v619: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:47:56.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:56 smithi135 ceph-mon[36839]: pgmap v619: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2024-06-02T15:47:56.966 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:56.967 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:57.079 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:57.079 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:57.079 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 74 -lt 100 ']' 2024-06-02T15:47:57.081 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 74 + 1 2024-06-02T15:47:57.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=75 2024-06-02T15:47:57.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:47:57.265 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:57.265 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:57.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:47:57.270 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:47:57.271 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:47:58.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:58 smithi090 ceph-mon[31595]: pgmap v620: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-06-02T15:47:58.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:58 smithi120 ceph-mon[36451]: pgmap v620: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-06-02T15:47:58.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:58 smithi135 ceph-mon[36839]: pgmap v620: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2024-06-02T15:47:58.875 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:47:58.875 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:47:59.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:47:59.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:47:59.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:47:59 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:47:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:47:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:47:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:47:59 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:47:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:47:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:47:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:47:59 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:47:59.629 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:47:59.629 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:47:59.696 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:59.696 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:47:59.696 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 75 -lt 100 ']' 2024-06-02T15:47:59.697 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 75 + 1 2024-06-02T15:47:59.987 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=76 2024-06-02T15:47:59.987 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:00.065 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:00.065 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:00.069 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:00.070 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:00.070 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:00.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:00 smithi120 ceph-mon[36451]: pgmap v621: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-06-02T15:48:00.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:00 smithi135 ceph-mon[36839]: pgmap v621: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-06-02T15:48:00.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:00 smithi090 ceph-mon[31595]: pgmap v621: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.349+0000 7fa79b200640 1 -- 172.21.15.90:0/3319321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c104b10 msgr2=0x7fa79c104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.349+0000 7fa79b200640 1 --2- 172.21.15.90:0/3319321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c104b10 0x7fa79c104f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009900 tx=0x7fa78c02a400 comp rx=0 tx=0).stop 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 -- 172.21.15.90:0/3319321178 shutdown_connections 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 --2- 172.21.15.90:0/3319321178 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c1066c0 0x7fa79c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 --2- 172.21.15.90:0/3319321178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 0x7fa79c106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 --2- 172.21.15.90:0/3319321178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c104b10 0x7fa79c104f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 -- 172.21.15.90:0/3319321178 >> 172.21.15.90:0/3319321178 conn(0x7fa79c100080 msgr2=0x7fa79c1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 -- 172.21.15.90:0/3319321178 shutdown_connections 2024-06-02T15:48:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.350+0000 7fa79b200640 1 -- 172.21.15.90:0/3319321178 wait complete. 2024-06-02T15:48:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.442+0000 7fa79b200640 1 Processor -- start 2024-06-02T15:48:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.443+0000 7fa79b200640 1 -- start start 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 0x7fa79c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 0x7fa79c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c110640 con 0x7fa79c105d00 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1104c0 con 0x7fa79c104b10 2024-06-02T15:48:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.482+0000 7fa79b200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1107c0 con 0x7fa79c1066c0 2024-06-02T15:48:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa799e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 0x7fa79c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa793e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 0x7fa79c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53528/0 (socket says 172.21.15.90:53528) 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 -- 172.21.15.90:0/3565416839 learned_addr learned my addr 172.21.15.90:0/3565416839 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa799e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 0x7fa79c19ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53800/0 (socket says 172.21.15.90:53800) 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 -- 172.21.15.90:0/3565416839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 msgr2=0x7fa79c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 0x7fa79c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 -- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 msgr2=0x7fa79c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 0x7fa79c19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.483+0000 7fa79a800640 1 -- 172.21.15.90:0/3565416839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c1a2960 con 0x7fa79c1066c0 2024-06-02T15:48:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.484+0000 7fa79a800640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa79400eb10 tx=0x7fa79400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.493+0000 7fa792a00640 1 -- 172.21.15.90:0/3565416839 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794004b20 con 0x7fa79c1066c0 2024-06-02T15:48:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.493+0000 7fa792a00640 1 -- 172.21.15.90:0/3565416839 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa794004cc0 con 0x7fa79c1066c0 2024-06-02T15:48:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.493+0000 7fa792a00640 1 -- 172.21.15.90:0/3565416839 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa794010670 con 0x7fa79c1066c0 2024-06-02T15:48:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.493+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c1a2c50 con 0x7fa79c1066c0 2024-06-02T15:48:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.504+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a31a0 con 0x7fa79c1066c0 2024-06-02T15:48:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.509+0000 7fa792a00640 1 -- 172.21.15.90:0/3565416839 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa794005000 con 0x7fa79c1066c0 2024-06-02T15:48:01.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.564+0000 7fa792a00640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 0x7fa7800b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.565+0000 7fa792a00640 1 -- 172.21.15.90:0/3565416839 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7940e6060 con 0x7fa79c1066c0 2024-06-02T15:48:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.565+0000 7fa799e00640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 0x7fa7800b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:01.567+0000 7fa799e00640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 0x7fa7800b4df0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa79c19c220 tx=0x7fa78c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:01.941 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:01.941 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.033+0000 7fa7a4078740 1 Processor -- start 2024-06-02T15:48:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 -- start start 2024-06-02T15:48:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 0x5582966ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558296656b90 con 0x5582966e6cf0 2024-06-02T15:48:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558296646730 con 0x5582966e76e0 2024-06-02T15:48:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.034+0000 7fa7a4078740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55829664a770 con 0x5582966ec850 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.035+0000 7fa799e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.035+0000 7fa799e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37850/0 (socket says 172.21.15.90:37850) 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.035+0000 7fa799e00640 1 -- 172.21.15.90:0/3401731343 learned_addr learned my addr 172.21.15.90:0/3401731343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.035+0000 7fa79a800640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.035+0000 7fa793e00640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 0x5582966ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 -- 172.21.15.90:0/3401731343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 msgr2=0x5582966eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 -- 172.21.15.90:0/3401731343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 msgr2=0x5582966ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 0x5582966ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 -- 172.21.15.90:0/3401731343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582965df8b0 con 0x5582966e6cf0 2024-06-02T15:48:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa793e00640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 0x5582966ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:48:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.036+0000 7fa799e00640 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa79c105b60 tx=0x7fa78c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3484b06114d2af2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.037+0000 7fa78b400640 1 -- 172.21.15.90:0/3401731343 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c04d070 con 0x5582966e6cf0 2024-06-02T15:48:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.037+0000 7fa78b400640 1 -- 172.21.15.90:0/3401731343 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa78c003ca0 con 0x5582966e6cf0 2024-06-02T15:48:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.037+0000 7fa78b400640 1 -- 172.21.15.90:0/3401731343 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c035050 con 0x5582966e6cf0 2024-06-02T15:48:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 -- 172.21.15.90:0/3401731343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 msgr2=0x5582966e7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa79c105b60 tx=0x7fa78c03e000 comp rx=0 tx=0).stop 2024-06-02T15:48:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 -- 172.21.15.90:0/3401731343 shutdown_connections 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e76e0 0x5582966ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3401731343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e7110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 -- 172.21.15.90:0/3401731343 >> 172.21.15.90:0/3401731343 conn(0x5582966e2360 msgr2=0x5582966e2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.038+0000 7fa7a4078740 1 -- 172.21.15.90:0/3401731343 shutdown_connections 2024-06-02T15:48:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.039+0000 7fa7a4078740 1 -- 172.21.15.90:0/3401731343 wait complete. 2024-06-02T15:48:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.121+0000 7fa7a4078740 1 -- start start 2024-06-02T15:48:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.163+0000 7fa7a4078740 1 Processor -- start 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 0x5582966f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55829632c790 con 0x5582966e76e0 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5582965eebc0 con 0x5582966e6cf0 2024-06-02T15:48:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.164+0000 7fa7a4078740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5582966ef7a0 con 0x5582966ec850 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53540/0 (socket says 172.21.15.90:53540) 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 -- 172.21.15.90:0/3071323673 learned_addr learned my addr 172.21.15.90:0/3071323673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa799e00640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa793e00640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 0x5582966f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 -- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 msgr2=0x5582966f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 -- 172.21.15.90:0/3071323673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 msgr2=0x5582966f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 0x5582966f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa79a800640 1 -- 172.21.15.90:0/3071323673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829635b210 con 0x5582966ec850 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.165+0000 7fa793e00640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 0x5582966f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.166+0000 7fa79a800640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7940a9520 tx=0x7fa794005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c027e487f93461 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.166+0000 7fa77fe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.166+0000 7fa77fe00640 1 -- 172.21.15.90:0/3071323673 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940c2070 con 0x5582966ec850 2024-06-02T15:48:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.166+0000 7fa77fe00640 1 -- 172.21.15.90:0/3071323673 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7940a76f0 con 0x5582966ec850 2024-06-02T15:48:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.193+0000 7fa7a4078740 10 client.0 _ll_register_callbacks cb 0x5582965cc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:02.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.230+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c003ca0 con 0x5582966ec850 2024-06-02T15:48:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.231+0000 7fa77fe00640 1 -- 172.21.15.90:0/3071323673 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7940a90a0 con 0x5582966ec850 2024-06-02T15:48:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.231+0000 7fa7a4078740 10 client.25669 fetch_fsmap learned FSMap version 28 2024-06-02T15:48:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.231+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=28}) -- 0x558296427620 con 0x5582966ec850 2024-06-02T15:48:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.433+0000 7fa77fe00640 1 -- 172.21.15.90:0/3071323673 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa7940a87f0 con 0x5582966ec850 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.433+0000 7fa7a4078740 10 client.25669 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.433+0000 7fa7a4078740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.433+0000 7fa7a4078740 20 client.25669 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.433+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582966fc3f0 0x5582966fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.434+0000 7fa7a4078740 4 client.25669 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.434+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558296658ae0 con 0x5582966fc3f0 2024-06-02T15:48:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.434+0000 7fa799e00640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582966fc3f0 0x5582966fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.435+0000 7fa799e00640 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582966fc3f0 0x5582966fe8b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e71f812ff178c0 server_cookie=ac68827250f90aa3 in_seq=0 out_seq=0 2024-06-02T15:48:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.435+0000 7fa77fe00640 10 client.25669 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.436+0000 7fa77fe00640 1 -- 172.21.15.90:0/3071323673 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558296658ae0 con 0x5582966fc3f0 2024-06-02T15:48:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.436+0000 7fa77fe00640 10 client.25669 handle_command_reply: tid=0 2024-06-02T15:48:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.437+0000 7fa7a4078740 1 client.25669 shutdown 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.437+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 msgr2=0x5582966f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.437+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7940a9520 tx=0x7fa794005570 comp rx=0 tx=0).stop 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 shutdown_connections 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582966fc3f0 0x5582966fe8b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966ec850 0x5582966f8000 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e76e0 0x5582966f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa77fe00640 0 client.25669 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 --2- 172.21.15.90:0/3071323673 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 >> 172.21.15.90:0/3071323673 conn(0x5582966e2360 msgr2=0x5582966e14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 shutdown_connections 2024-06-02T15:48:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 1 -- 172.21.15.90:0/3071323673 wait complete. 2024-06-02T15:48:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.438+0000 7fa7a4078740 20 client.25669 trim_cache size 0 max 16384 2024-06-02T15:48:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.472+0000 7fa7a4078740 1 Processor -- start 2024-06-02T15:48:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.472+0000 7fa7a4078740 1 -- start start 2024-06-02T15:48:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1affd0 con 0x5582966e6cf0 2024-06-02T15:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ad800 con 0x5582966ec850 2024-06-02T15:48:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa7a4078740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ab650 con 0x5582966e76e0 2024-06-02T15:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa79a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa79a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53820/0 (socket says 172.21.15.90:53820) 2024-06-02T15:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa79a800640 1 -- 172.21.15.90:0/4276042029 learned_addr learned my addr 172.21.15.90:0/4276042029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa793e00640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.473+0000 7fa799e00640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 -- 172.21.15.90:0/4276042029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 msgr2=0x5582966f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 -- 172.21.15.90:0/4276042029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 msgr2=0x5582966e8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 -- 172.21.15.90:0/4276042029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582966e6930 con 0x5582966ec850 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa793e00640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa79a800640 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7940a9240 tx=0x7fa7940ad000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=621ed6f224e9a4d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.474+0000 7fa78aa00640 1 -- 172.21.15.90:0/4276042029 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940c2070 con 0x5582966ec850 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.475+0000 7fa78aa00640 1 -- 172.21.15.90:0/4276042029 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa78c003ca0 con 0x5582966ec850 2024-06-02T15:48:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.475+0000 7fa78aa00640 1 -- 172.21.15.90:0/4276042029 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940ad940 con 0x5582966ec850 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- 172.21.15.90:0/4276042029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 msgr2=0x5582966e86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa7940a9240 tx=0x7fa7940ad000 comp rx=0 tx=0).stop 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- 172.21.15.90:0/4276042029 shutdown_connections 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966ec850 0x5582966e86c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4276042029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966e6cf0 0x5582966e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- 172.21.15.90:0/4276042029 >> 172.21.15.90:0/4276042029 conn(0x5582966e2360 msgr2=0x5582966e5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- 172.21.15.90:0/4276042029 shutdown_connections 2024-06-02T15:48:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- 172.21.15.90:0/4276042029 wait complete. 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.476+0000 7fa7a4078740 1 -- start start 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 Processor -- start 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966ec850 0x5582966f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1abcc0 con 0x5582966ec850 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ad130 con 0x5582966e6cf0 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.477+0000 7fa7a4078740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ac390 con 0x5582966e76e0 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa7a4078740 10 client.0 _ll_register_callbacks cb 0x7fa794004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:53822/0 (socket says 172.21.15.90:53822) 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 -- 172.21.15.90:0/4192668618 learned_addr learned my addr 172.21.15.90:0/4192668618 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa79a800640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966ec850 0x5582966f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa793e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 -- 172.21.15.90:0/4192668618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 msgr2=0x5582966f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 -- 172.21.15.90:0/4192668618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966ec850 msgr2=0x5582966f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966ec850 0x5582966f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.478+0000 7fa799e00640 1 -- 172.21.15.90:0/4192668618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582966f70a0 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa799e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa78c05efd0 tx=0x7fa78c004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f69afaf29a097f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa77fe00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c04d040 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa78c003ca0 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa78c003a00 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa78c035040 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa7a4078740 10 client.25712 fetch_fsmap learned FSMap version 28 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.479+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558296767090 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa78c003ca0 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa78c0526f0 con 0x5582966e6cf0 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 10 client.25712 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 20 client.25712 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558296767470 0x558296769890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 4 client.25712 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582966522c0 con 0x558296767470 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.480+0000 7fa793e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558296767470 0x558296769890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.481+0000 7fa793e00640 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558296767470 0x558296769890 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e2760dbfe212d5 server_cookie=11397c86e5576758 in_seq=0 out_seq=0 2024-06-02T15:48:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.481+0000 7fa77fe00640 10 client.25712 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:02.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:02 smithi090 ceph-mon[31595]: pgmap v622: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.1 KiB/s wr, 7 op/s 2024-06-02T15:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.503+0000 7fa77fe00640 1 -- 172.21.15.90:0/4192668618 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5582966522c0 con 0x558296767470 2024-06-02T15:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.503+0000 7fa77fe00640 10 client.25712 handle_command_reply: tid=0 2024-06-02T15:48:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.503+0000 7fa7a4078740 1 client.25712 shutdown 2024-06-02T15:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 msgr2=0x5582966f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa78c05efd0 tx=0x7fa78c004590 comp rx=0 tx=0).stop 2024-06-02T15:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 shutdown_connections 2024-06-02T15:48:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x558296767470 0x558296769890 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582966ec850 0x5582966f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa77fe00640 0 client.25712 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582966e76e0 0x5582966f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 --2- 172.21.15.90:0/4192668618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582966e6cf0 0x5582966f55b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 >> 172.21.15.90:0/4192668618 conn(0x5582966e2360 msgr2=0x5582966e8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 shutdown_connections 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 1 -- 172.21.15.90:0/4192668618 wait complete. 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.504+0000 7fa7a4078740 20 client.25712 trim_cache size 0 max 16384 2024-06-02T15:48:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:48:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:48:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:48:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.505+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 msgr2=0x7fa7800b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.505+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 0x7fa7800b4df0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa79c19c220 tx=0x7fa78c0314a0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.505+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 msgr2=0x7fa79c1a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.505+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa79400eb10 tx=0x7fa79400c2d0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 shutdown_connections 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7800b2930 0x7fa7800b4df0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa79c1066c0 0x7fa79c1a22c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa79c105d00 0x7fa79c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 --2- 172.21.15.90:0/3565416839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa79c104b10 0x7fa79c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 >> 172.21.15.90:0/3565416839 conn(0x7fa79c100080 msgr2=0x7fa79c109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 shutdown_connections 2024-06-02T15:48:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:02.506+0000 7fa79b200640 1 -- 172.21.15.90:0/3565416839 wait complete. 2024-06-02T15:48:02.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:02 smithi120 ceph-mon[36451]: pgmap v622: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.1 KiB/s wr, 7 op/s 2024-06-02T15:48:02.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:02 smithi135 ceph-mon[36839]: pgmap v622: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.1 KiB/s wr, 7 op/s 2024-06-02T15:48:02.700 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:02.700 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:02.747 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:02.747 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:02.747 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 76 -lt 100 ']' 2024-06-02T15:48:02.748 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 76 + 1 2024-06-02T15:48:02.886 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=77 2024-06-02T15:48:02.887 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:03.099 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:03.099 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:03.104 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:03.105 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:03.105 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:04.423 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:04 smithi090 ceph-mon[31595]: pgmap v623: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-02T15:48:04.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:04 smithi120 ceph-mon[36451]: pgmap v623: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-02T15:48:04.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:04 smithi135 ceph-mon[36839]: pgmap v623: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 9 op/s 2024-06-02T15:48:04.908 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:04.908 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:05.424 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:05.425 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:05.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:05.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:05.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 77 -lt 100 ']' 2024-06-02T15:48:05.468 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 77 + 1 2024-06-02T15:48:05.644 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=78 2024-06-02T15:48:05.644 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:05.791 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:05.791 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:05.796 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:05.797 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:05.797 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:06.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:06 smithi120 ceph-mon[36451]: pgmap v624: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-06-02T15:48:06.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:06 smithi135 ceph-mon[36839]: pgmap v624: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-06-02T15:48:06.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:06 smithi090 ceph-mon[31595]: pgmap v624: 129 pgs: 129 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-06-02T15:48:07.808 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:07.809 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:08.449 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:08.450 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:08.481 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:08.481 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:08.481 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 78 -lt 100 ']' 2024-06-02T15:48:08.482 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 78 + 1 2024-06-02T15:48:08.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:08 smithi120 ceph-mon[36451]: pgmap v625: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:08.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:08 smithi135 ceph-mon[36839]: pgmap v625: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:08.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:08 smithi090 ceph-mon[31595]: pgmap v625: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:08.711 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=79 2024-06-02T15:48:08.711 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:08.849 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:08.850 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:08.854 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:08.855 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:08.856 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:09.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.005+0000 7f8f68600700 1 -- 172.21.15.90:0/582046510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 msgr2=0x7f8f6006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.005+0000 7f8f68600700 1 --2- 172.21.15.90:0/582046510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6006bc10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009990 tx=0x7f8f54009ca0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 -- 172.21.15.90:0/582046510 shutdown_connections 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 --2- 172.21.15.90:0/582046510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6010fe90 0x7f8f60112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 --2- 172.21.15.90:0/582046510 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6006bc10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 --2- 172.21.15.90:0/582046510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6006c150 0x7f8f6006c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 -- 172.21.15.90:0/582046510 >> 172.21.15.90:0/582046510 conn(0x7f8f600793f0 msgr2=0x7f8f60079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 -- 172.21.15.90:0/582046510 shutdown_connections 2024-06-02T15:48:09.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.006+0000 7f8f68600700 1 -- 172.21.15.90:0/582046510 wait complete. 2024-06-02T15:48:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.007+0000 7f8f68600700 1 Processor -- start 2024-06-02T15:48:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.007+0000 7f8f68600700 1 -- start start 2024-06-02T15:48:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 0x7f8f601a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60115950 con 0x7f8f6010fe90 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f601157d0 con 0x7f8f60104450 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f68600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60115ad0 con 0x7f8f6006c150 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f65c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f65c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53584/0 (socket says 172.21.15.90:53584) 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f65c00700 1 -- 172.21.15.90:0/2598098706 learned_addr learned my addr 172.21.15.90:0/2598098706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f65200700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:09.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.008+0000 7f8f66600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 0x7f8f601a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 -- 172.21.15.90:0/2598098706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 msgr2=0x7f8f6019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 -- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 msgr2=0x7f8f601a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 0x7f8f601a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 -- 172.21.15.90:0/2598098706 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601a0e50 con 0x7f8f6006c150 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f66600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 0x7f8f601a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.009+0000 7f8f65c00700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005910 tx=0x7f8f5400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.010+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54027070 con 0x7f8f6006c150 2024-06-02T15:48:09.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.010+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f54022470 con 0x7f8f6006c150 2024-06-02T15:48:09.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.010+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54017650 con 0x7f8f6006c150 2024-06-02T15:48:09.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.010+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f601a10e0 con 0x7f8f6006c150 2024-06-02T15:48:09.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.010+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601a89c0 con 0x7f8f6006c150 2024-06-02T15:48:09.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.011+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f60102a80 con 0x7f8f6006c150 2024-06-02T15:48:09.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.013+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8f5402a070 con 0x7f8f6006c150 2024-06-02T15:48:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.015+0000 7f8f5e000700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 0x7f8f480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.015+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8f540e3550 con 0x7f8f6006c150 2024-06-02T15:48:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.015+0000 7f8f65200700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 0x7f8f480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:09.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.016+0000 7f8f65200700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 0x7f8f480b4dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f60079cf0 tx=0x7f8f4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:09.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.017+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f54013070 con 0x7f8f6006c150 2024-06-02T15:48:09.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.310+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f601a8b90 con 0x7f8f6006c150 2024-06-02T15:48:09.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.311+0000 7f8f5e000700 1 -- 172.21.15.90:0/2598098706 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11879 (secure 0 0 0) 0x7f8f540aaef0 con 0x7f8f6006c150 2024-06-02T15:48:09.309 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:48:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:46:44.163919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6837/1676077338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1676077338},{"type":"v1","addr":"172.21.15.120:6837","nonce":1676077338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 msgr2=0x7f8f480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 0x7f8f480b4dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f60079cf0 tx=0x7f8f4c00b410 comp rx=0 tx=0).stop 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 msgr2=0x7f8f6019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005910 tx=0x7f8f5400b670 comp rx=0 tx=0).stop 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 shutdown_connections 2024-06-02T15:48:09.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8f480b2900 0x7f8f480b4dc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f6010fe90 0x7f8f601a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f6006c150 0x7f8f6019be40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 --2- 172.21.15.90:0/2598098706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f60104450 0x7f8f6019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 >> 172.21.15.90:0/2598098706 conn(0x7f8f600793f0 msgr2=0x7f8f60101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 shutdown_connections 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:48:09.314+0000 7f8f68600700 1 -- 172.21.15.90:0/2598098706 wait complete. 2024-06-02T15:48:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 28 2024-06-02T15:48:09.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:09 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2598098706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:48:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:09 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2598098706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:48:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:09 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2598098706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:48:10.508 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:10.508 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:10.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:10 smithi090 ceph-mon[31595]: pgmap v626: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-06-02T15:48:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:10 smithi120 ceph-mon[36451]: pgmap v626: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-06-02T15:48:10.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:10 smithi135 ceph-mon[36839]: pgmap v626: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2024-06-02T15:48:11.049 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:11.050 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:11.083 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:11.083 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:11.083 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 79 -lt 100 ']' 2024-06-02T15:48:11.084 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 79 + 1 2024-06-02T15:48:11.177 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=80 2024-06-02T15:48:11.177 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:11.232 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:11.232 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:11.236 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:11.237 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:11.238 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:12.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:12 smithi090 ceph-mon[31595]: pgmap v627: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-06-02T15:48:12.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:12 smithi120 ceph-mon[36451]: pgmap v627: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-06-02T15:48:12.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:12 smithi135 ceph-mon[36839]: pgmap v627: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-06-02T15:48:12.708 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:12.708 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:13.207 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:13.208 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:13.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:13.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:13.240 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 80 -lt 100 ']' 2024-06-02T15:48:13.241 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 80 + 1 2024-06-02T15:48:13.335 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=81 2024-06-02T15:48:13.336 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:13.440 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:13.441 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:13.445 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:13.446 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:13.447 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:14.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:14 smithi090 ceph-mon[31595]: pgmap v628: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-02T15:48:14.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:48:14.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:14 smithi120 ceph-mon[36451]: pgmap v628: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-02T15:48:14.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:14 smithi135 ceph-mon[36839]: pgmap v628: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-06-02T15:48:15.024 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:15.024 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:15.324 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:15.325 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:15.325 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 81 -lt 100 ']' 2024-06-02T15:48:15.325 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 81 + 1 2024-06-02T15:48:15.478 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=82 2024-06-02T15:48:15.478 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:15.514 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:15.515 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:15.520 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:15.521 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:15.521 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:16.866 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:16.866 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:17.163 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:17.164 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:17.223 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:17.224 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:17.224 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 82 -lt 100 ']' 2024-06-02T15:48:17.225 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 82 + 1 2024-06-02T15:48:17.344 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=83 2024-06-02T15:48:17.344 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:17.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:17 smithi090 ceph-mon[31595]: pgmap v629: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-06-02T15:48:17.428 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:17.428 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:17.432 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:17.434 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:17.434 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:17 smithi120 ceph-mon[36451]: pgmap v629: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-06-02T15:48:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:17 smithi135 ceph-mon[36839]: pgmap v629: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-06-02T15:48:19.249 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:19.249 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:19.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:19 smithi090 ceph-mon[31595]: pgmap v630: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-02T15:48:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:19 smithi120 ceph-mon[36451]: pgmap v630: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-02T15:48:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:19 smithi135 ceph-mon[36839]: pgmap v630: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-06-02T15:48:19.858 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:19.858 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:19.923 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:19.923 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:19.923 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 83 -lt 100 ']' 2024-06-02T15:48:19.924 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 83 + 1 2024-06-02T15:48:20.127 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=84 2024-06-02T15:48:20.127 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:20.196 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:20.196 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:20.201 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:20.202 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:20.202 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:20.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:20 smithi120 ceph-mon[36451]: pgmap v631: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-06-02T15:48:20.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:20 smithi135 ceph-mon[36839]: pgmap v631: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-06-02T15:48:20.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:20 smithi090 ceph-mon[31595]: pgmap v631: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.010+0000 7f6869800640 1 -- 172.21.15.90:0/423635353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6864100700 msgr2=0x7f6864100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.010+0000 7f6869800640 1 --2- 172.21.15.90:0/423635353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6864100700 0x7f6864100b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6854009900 tx=0x7f685402a400 comp rx=0 tx=0).stop 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 -- 172.21.15.90:0/423635353 shutdown_connections 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 --2- 172.21.15.90:0/423635353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 0x7f686410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 --2- 172.21.15.90:0/423635353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f68641010f0 0x7f6864108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 --2- 172.21.15.90:0/423635353 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6864100700 0x7f6864100b20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 -- 172.21.15.90:0/423635353 >> 172.21.15.90:0/423635353 conn(0x7f68640fc270 msgr2=0x7f68640fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 -- 172.21.15.90:0/423635353 shutdown_connections 2024-06-02T15:48:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.011+0000 7f6869800640 1 -- 172.21.15.90:0/423635353 wait complete. 2024-06-02T15:48:22.014 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:22.015 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.105+0000 7f6869800640 1 Processor -- start 2024-06-02T15:48:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.106+0000 7f6869800640 1 -- start start 2024-06-02T15:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68641010f0 0x7f686419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 0x7f68641a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f686410e620 con 0x7f6864100700 2024-06-02T15:48:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6862a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6862a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33546/0 (socket says 172.21.15.90:33546) 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6862a00640 1 -- 172.21.15.90:0/1901727750 learned_addr learned my addr 172.21.15.90:0/1901727750 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f686410e4a0 con 0x7f68641010f0 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862000640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68641010f0 0x7f686419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6863400640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 0x7f68641a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.107+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f686410e7a0 con 0x7f6864108780 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862a00640 1 -- 172.21.15.90:0/1901727750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 msgr2=0x7f68641a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862a00640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 0x7f68641a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862a00640 1 -- 172.21.15.90:0/1901727750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68641010f0 msgr2=0x7f686419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862a00640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68641010f0 0x7f686419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.108+0000 7f6862a00640 1 -- 172.21.15.90:0/1901727750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68641a2ad0 con 0x7f6864100700 2024-06-02T15:48:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.109+0000 7f6862a00640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6854005ab0 tx=0x7f6854003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.128+0000 7f685b400640 1 -- 172.21.15.90:0/1901727750 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685403f070 con 0x7f6864100700 2024-06-02T15:48:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.128+0000 7f685b400640 1 -- 172.21.15.90:0/1901727750 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6854031c50 con 0x7f6864100700 2024-06-02T15:48:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.128+0000 7f685b400640 1 -- 172.21.15.90:0/1901727750 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854035030 con 0x7f6864100700 2024-06-02T15:48:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.128+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68641a2d60 con 0x7f6864100700 2024-06-02T15:48:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.133+0000 7f685b400640 1 -- 172.21.15.90:0/1901727750 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6854042070 con 0x7f6864100700 2024-06-02T15:48:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.135+0000 7f685b400640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 0x7f68480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.135+0000 7f6862000640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 0x7f68480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.136+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68641a31a0 con 0x7f6864100700 2024-06-02T15:48:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.137+0000 7f685b400640 1 -- 172.21.15.90:0/1901727750 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f68540c5cf0 con 0x7f6864100700 2024-06-02T15:48:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.137+0000 7f6862000640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 0x7f68480b4eb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f686419c3c0 tx=0x7f6850005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.622+0000 7f686c5d1740 1 Processor -- start 2024-06-02T15:48:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.623+0000 7f686c5d1740 1 -- start start 2024-06-02T15:48:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.623+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.623+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc773020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.623+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 0x55b8dc775b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f686c5d1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc6ddb90 con 0x55b8dc76e520 2024-06-02T15:48:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f686c5d1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc6cd730 con 0x55b8dc76db30 2024-06-02T15:48:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f686c5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc6d1770 con 0x55b8dc773690 2024-06-02T15:48:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc773020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc773020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33552/0 (socket says 172.21.15.90:33552) 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862000640 1 -- 172.21.15.90:0/856269473 learned_addr learned my addr 172.21.15.90:0/856269473 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36496/0 (socket says 172.21.15.90:36496) 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6863400640 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 0x55b8dc775b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862a00640 1 -- 172.21.15.90:0/856269473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 msgr2=0x55b8dc775b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.624+0000 7f6862a00640 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 0x55b8dc775b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f6862a00640 1 -- 172.21.15.90:0/856269473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 msgr2=0x55b8dc773020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f6862a00640 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc773020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f6862a00640 1 -- 172.21.15.90:0/856269473 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8dc6668b0 con 0x55b8dc76db30 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f6863400640 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 0x55b8dc775b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f6862a00640 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68641a2f30 tx=0x7f68540c8e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e2ea1aaf65931d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.625+0000 7f684fe00640 1 -- 172.21.15.90:0/856269473 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68540c9700 con 0x55b8dc76db30 2024-06-02T15:48:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.626+0000 7f684fe00640 1 -- 172.21.15.90:0/856269473 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68540cdab0 con 0x55b8dc76db30 2024-06-02T15:48:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.626+0000 7f684fe00640 1 -- 172.21.15.90:0/856269473 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68540d2330 con 0x55b8dc76db30 2024-06-02T15:48:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 -- 172.21.15.90:0/856269473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 msgr2=0x55b8dc76df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f68641a2f30 tx=0x7f68540c8e70 comp rx=0 tx=0).stop 2024-06-02T15:48:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 -- 172.21.15.90:0/856269473 shutdown_connections 2024-06-02T15:48:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc773690 0x55b8dc775b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc773020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 --2- 172.21.15.90:0/856269473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76db30 0x55b8dc76df50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 -- 172.21.15.90:0/856269473 >> 172.21.15.90:0/856269473 conn(0x55b8dc7691a0 msgr2=0x55b8dc7695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 -- 172.21.15.90:0/856269473 shutdown_connections 2024-06-02T15:48:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.627+0000 7f686c5d1740 1 -- 172.21.15.90:0/856269473 wait complete. 2024-06-02T15:48:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.658+0000 7f686c5d1740 1 -- start start 2024-06-02T15:48:22.682 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:22.682 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.688+0000 7f686c5d1740 1 Processor -- start 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 0x55b8dc777670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76e520 0x55b8dc777bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc3b3790 con 0x55b8dc76db30 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc675bc0 con 0x55b8dc773690 2024-06-02T15:48:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f686c5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b8dc776400 con 0x55b8dc76e520 2024-06-02T15:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6862a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 0x55b8dc777670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6863400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6863400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36510/0 (socket says 172.21.15.90:36510) 2024-06-02T15:48:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6862a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 0x55b8dc777670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33556/0 (socket says 172.21.15.90:33556) 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6863400640 1 -- 172.21.15.90:0/1118249847 learned_addr learned my addr 172.21.15.90:0/1118249847 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.689+0000 7f6862000640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76e520 0x55b8dc777bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 -- 172.21.15.90:0/1118249847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76e520 msgr2=0x55b8dc777bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76e520 0x55b8dc777bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 -- 172.21.15.90:0/1118249847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 msgr2=0x55b8dc777670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 0x55b8dc777670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 -- 172.21.15.90:0/1118249847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8dc3e2210 con 0x55b8dc773690 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f6863400640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f685c012d20 tx=0x7f685c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a4403a4658e492 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f684cc00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.690+0000 7f684cc00640 1 -- 172.21.15.90:0/1118249847 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c00fe60 con 0x55b8dc773690 2024-06-02T15:48:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.691+0000 7f684cc00640 1 -- 172.21.15.90:0/1118249847 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f685c012110 con 0x55b8dc773690 2024-06-02T15:48:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.697+0000 7f686c5d1740 10 client.0 _ll_register_callbacks cb 0x55b8dc653400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.698+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68540cdab0 con 0x55b8dc773690 2024-06-02T15:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.699+0000 7f684cc00640 1 -- 172.21.15.90:0/1118249847 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f685c003970 con 0x55b8dc773690 2024-06-02T15:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.699+0000 7f686c5d1740 10 client.25730 fetch_fsmap learned FSMap version 28 2024-06-02T15:48:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.699+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b8dc4ae620 con 0x55b8dc773690 2024-06-02T15:48:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f684cc00640 1 -- 172.21.15.90:0/1118249847 <== mon.2 v2:172.21.15.120:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f685c003b10 con 0x55b8dc773690 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 10 client.25730 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 20 client.25730 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc782ee0 0x55b8dc7853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 4 client.25730 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:48:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8dc6dfae0 con 0x55b8dc782ee0 2024-06-02T15:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.719+0000 7f6862a00640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc782ee0 0x55b8dc7853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.720+0000 7f6862a00640 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc782ee0 0x55b8dc7853a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f516b41976bd6cc server_cookie=5b45edb0aad9ac63 in_seq=0 out_seq=0 2024-06-02T15:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.720+0000 7f684cc00640 10 client.25730 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f684cc00640 1 -- 172.21.15.90:0/1118249847 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8dc6dfae0 con 0x55b8dc782ee0 2024-06-02T15:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f684cc00640 10 client.25730 handle_command_reply: tid=0 2024-06-02T15:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 client.25730 shutdown 2024-06-02T15:48:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 msgr2=0x55b8dc77ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f685c012d20 tx=0x7f685c00f790 comp rx=0 tx=0).stop 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 shutdown_connections 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc782ee0 0x55b8dc7853a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc77ebf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76e520 0x55b8dc777bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 --2- 172.21.15.90:0/1118249847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76db30 0x55b8dc777670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f684cc00640 0 client.25730 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.721+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 >> 172.21.15.90:0/1118249847 conn(0x55b8dc7691a0 msgr2=0x55b8dc768310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.722+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 shutdown_connections 2024-06-02T15:48:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.722+0000 7f686c5d1740 1 -- 172.21.15.90:0/1118249847 wait complete. 2024-06-02T15:48:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.722+0000 7f686c5d1740 20 client.25730 trim_cache size 0 max 16384 2024-06-02T15:48:22.762 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:22.762 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:22.762 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 84 -lt 100 ']' 2024-06-02T15:48:22.764 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 84 + 1 2024-06-02T15:48:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.772+0000 7f686c5d1740 1 Processor -- start 2024-06-02T15:48:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 -- start start 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc77eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc76f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b0200 con 0x55b8dc76e520 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68641ada30 con 0x55b8dc773690 2024-06-02T15:48:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.773+0000 7f686c5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68641ab880 con 0x55b8dc76db30 2024-06-02T15:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.774+0000 7f6862a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.774+0000 7f6862a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60950/0 (socket says 172.21.15.90:60950) 2024-06-02T15:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.774+0000 7f6862a00640 1 -- 172.21.15.90:0/3538504068 learned_addr learned my addr 172.21.15.90:0/3538504068 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.774+0000 7f6863400640 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc76f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.774+0000 7f6862000640 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc77eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.775+0000 7f6862a00640 1 -- 172.21.15.90:0/3538504068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 msgr2=0x55b8dc76f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.775+0000 7f6862a00640 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc76f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.775+0000 7f6862a00640 1 -- 172.21.15.90:0/3538504068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 msgr2=0x55b8dc77eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.775+0000 7f6862a00640 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc77eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.775+0000 7f6862a00640 1 -- 172.21.15.90:0/3538504068 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8dc7633f0 con 0x55b8dc76db30 2024-06-02T15:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.776+0000 7f6862a00640 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68540c9070 tx=0x7f68540c5990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd8a103c704a9e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.776+0000 7f684f400640 1 -- 172.21.15.90:0/3538504068 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685403b4e0 con 0x55b8dc76db30 2024-06-02T15:48:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.776+0000 7f684f400640 1 -- 172.21.15.90:0/3538504068 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68540d8070 con 0x55b8dc76db30 2024-06-02T15:48:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.776+0000 7f684f400640 1 -- 172.21.15.90:0/3538504068 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68540d2330 con 0x55b8dc76db30 2024-06-02T15:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.777+0000 7f686c5d1740 1 -- 172.21.15.90:0/3538504068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 msgr2=0x55b8dc76efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.777+0000 7f686c5d1740 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68540c9070 tx=0x7f68540c5990 comp rx=0 tx=0).stop 2024-06-02T15:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 -- 172.21.15.90:0/3538504068 shutdown_connections 2024-06-02T15:48:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc773690 0x55b8dc76f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc76e520 0x55b8dc77eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 --2- 172.21.15.90:0/3538504068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc76efc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 -- 172.21.15.90:0/3538504068 >> 172.21.15.90:0/3538504068 conn(0x55b8dc7691a0 msgr2=0x55b8dc76c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 -- 172.21.15.90:0/3538504068 shutdown_connections 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 -- 172.21.15.90:0/3538504068 wait complete. 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.778+0000 7f686c5d1740 1 -- start start 2024-06-02T15:48:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.779+0000 7f686c5d1740 1 Processor -- start 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.779+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.779+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76e520 0x55b8dc77c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f686c5d1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 0x55b8dc77d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f686c5d1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f68641abef0 con 0x55b8dc773690 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f686c5d1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f68641ad360 con 0x55b8dc76e520 2024-06-02T15:48:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f686c5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68641ac5c0 con 0x55b8dc76db30 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f686c5d1740 10 client.0 _ll_register_callbacks cb 0x7f685c009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60962/0 (socket says 172.21.15.90:60962) 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 -- 172.21.15.90:0/167325965 learned_addr learned my addr 172.21.15.90:0/167325965 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6863400640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 0x55b8dc77d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862000640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76e520 0x55b8dc77c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 -- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76e520 msgr2=0x55b8dc77c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76e520 0x55b8dc77c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 -- 172.21.15.90:0/167325965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 msgr2=0x55b8dc77d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 0x55b8dc77d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6862a00640 1 -- 172.21.15.90:0/167325965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8dc778510 con 0x55b8dc76db30 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.780+0000 7f6863400640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 0x55b8dc77d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:48:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f6862a00640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68540c6970 tx=0x7f68540c8e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f501b89933f9b653 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f684cc00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685403b4e0 con 0x55b8dc76db30 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f68540d20d0 con 0x55b8dc76db30 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68540d8070 con 0x55b8dc76db30 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68540c9700 con 0x55b8dc76db30 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f686c5d1740 10 client.25699 fetch_fsmap learned FSMap version 28 2024-06-02T15:48:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.781+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8dc778510 con 0x55b8dc76db30 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68540d8070 con 0x55b8dc76db30 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f68540318c0 con 0x55b8dc76db30 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 10 client.25699 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 20 client.25699 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc7edb80 0x55b8dc7eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 4 client.25699 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:48:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.782+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8dc6d92c0 con 0x55b8dc7edb80 2024-06-02T15:48:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.783+0000 7f6862000640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc7edb80 0x55b8dc7eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.783+0000 7f6862000640 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc7edb80 0x55b8dc7eff70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a7783834ef6ac3 server_cookie=125143cc612f930e in_seq=0 out_seq=0 2024-06-02T15:48:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.783+0000 7f684cc00640 10 client.25699 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.802+0000 7f684cc00640 1 -- 172.21.15.90:0/167325965 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b8dc6d92c0 con 0x55b8dc7edb80 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.802+0000 7f684cc00640 10 client.25699 handle_command_reply: tid=0 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.802+0000 7f686c5d1740 1 client.25699 shutdown 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 msgr2=0x55b8dc77bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68540c6970 tx=0x7f68540c8e70 comp rx=0 tx=0).stop 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 shutdown_connections 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b8dc7edb80 0x55b8dc7eff70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8dc773690 0x55b8dc77d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b8dc76e520 0x55b8dc77c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 --2- 172.21.15.90:0/167325965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8dc76db30 0x55b8dc77bf90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f684cc00640 0 client.25699 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 >> 172.21.15.90:0/167325965 conn(0x55b8dc7691a0 msgr2=0x55b8dc76efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 shutdown_connections 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 1 -- 172.21.15.90:0/167325965 wait complete. 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.803+0000 7f686c5d1740 20 client.25699 trim_cache size 0 max 16384 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:48:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 msgr2=0x7f68480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 0x7f68480b4eb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f686419c3c0 tx=0x7f6850005f90 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 msgr2=0x7f686419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6854005ab0 tx=0x7f6854003b90 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 shutdown_connections 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f68480b29f0 0x7f68480b4eb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864108780 0x7f68641a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f68641010f0 0x7f686419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 --2- 172.21.15.90:0/1901727750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6864100700 0x7f686419aeb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 >> 172.21.15.90:0/1901727750 conn(0x7f68640fc270 msgr2=0x7f68640fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 shutdown_connections 2024-06-02T15:48:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:22.847+0000 7f6869800640 1 -- 172.21.15.90:0/1901727750 wait complete. 2024-06-02T15:48:22.865 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=85 2024-06-02T15:48:22.865 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:23.073 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:23.073 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:23.080 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:23.081 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:23.082 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:23 smithi090 ceph-mon[31595]: pgmap v632: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:48:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:23 smithi120 ceph-mon[36451]: pgmap v632: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:48:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:23 smithi135 ceph-mon[36839]: pgmap v632: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:48:24.940 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:24.941 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:25.366 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:25.366 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:25.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:25 smithi090 ceph-mon[31595]: pgmap v633: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:25.398 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:25.398 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:25.398 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 85 -lt 100 ']' 2024-06-02T15:48:25.398 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 85 + 1 2024-06-02T15:48:25.502 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=86 2024-06-02T15:48:25.502 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:25.548 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:25.548 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:25.553 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:25.554 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:25.555 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:25 smithi120 ceph-mon[36451]: pgmap v633: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:25 smithi135 ceph-mon[36839]: pgmap v633: 129 pgs: 129 active+clean; 2.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:26.459 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:26 smithi120 ceph-mon[36451]: pgmap v634: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-06-02T15:48:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:26 smithi135 ceph-mon[36839]: pgmap v634: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-06-02T15:48:26.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:26 smithi090 ceph-mon[31595]: pgmap v634: 129 pgs: 129 active+clean; 2.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-06-02T15:48:27.240 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:27.241 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:27.660 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:27.660 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:27.718 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:27.718 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:27.718 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 86 -lt 100 ']' 2024-06-02T15:48:27.719 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 86 + 1 2024-06-02T15:48:27.846 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=87 2024-06-02T15:48:27.846 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:27.920 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:27.921 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:27.925 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:27.926 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:27.926 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:29.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:29 smithi090 ceph-mon[31595]: pgmap v635: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:29 smithi120 ceph-mon[36451]: pgmap v635: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:29 smithi135 ceph-mon[36839]: pgmap v635: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-06-02T15:48:29.682 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:29.682 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:30.474 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:30.474 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:30.512 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:30.512 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:30.512 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 87 -lt 100 ']' 2024-06-02T15:48:30.513 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 87 + 1 2024-06-02T15:48:30.583 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=88 2024-06-02T15:48:30.583 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:30.678 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:30.679 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:30.684 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:30.685 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:30.685 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:31.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:31 smithi090 ceph-mon[31595]: pgmap v636: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:31 smithi120 ceph-mon[36451]: pgmap v636: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:31 smithi135 ceph-mon[36839]: pgmap v636: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:32.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:32 smithi090 ceph-mon[31595]: pgmap v637: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2024-06-02T15:48:32.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:32 smithi120 ceph-mon[36451]: pgmap v637: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2024-06-02T15:48:32.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:32 smithi135 ceph-mon[36839]: pgmap v637: 129 pgs: 129 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 12 op/s 2024-06-02T15:48:32.623 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:32.624 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:33.257 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:33.257 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:33.327 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:33.327 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:33.327 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 88 -lt 100 ']' 2024-06-02T15:48:33.329 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 88 + 1 2024-06-02T15:48:33.491 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=89 2024-06-02T15:48:33.491 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:33.640 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:33.640 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:33.644 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:33.646 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:33.647 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:35.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:35 smithi090 ceph-mon[31595]: pgmap v638: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:48:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:35 smithi120 ceph-mon[36451]: pgmap v638: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:48:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:35 smithi135 ceph-mon[36839]: pgmap v638: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-06-02T15:48:35.672 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:35.672 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:36.178 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:36.178 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:36.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:36.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:36.222 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 89 -lt 100 ']' 2024-06-02T15:48:36.223 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 89 + 1 2024-06-02T15:48:36.360 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=90 2024-06-02T15:48:36.360 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:36.537 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:36.537 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:36.541 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:36.542 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:36.543 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:36 smithi120 ceph-mon[36451]: pgmap v639: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:36 smithi135 ceph-mon[36839]: pgmap v639: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:36 smithi090 ceph-mon[31595]: pgmap v639: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-06-02T15:48:38.165 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:38.165 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:38.690 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:38.690 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:38.727 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:38.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:38.728 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 90 -lt 100 ']' 2024-06-02T15:48:38.729 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 90 + 1 2024-06-02T15:48:38.834 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=91 2024-06-02T15:48:38.835 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:38.912 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:38.912 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:38.917 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:38.918 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:38.918 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:39.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:39 smithi090 ceph-mon[31595]: pgmap v640: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:39 smithi120 ceph-mon[36451]: pgmap v640: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:39 smithi135 ceph-mon[36839]: pgmap v640: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-06-02T15:48:40.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:40 smithi090 ceph-mon[31595]: pgmap v641: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:48:40.556 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:40.556 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:40 smithi120 ceph-mon[36451]: pgmap v641: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:48:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:40 smithi135 ceph-mon[36839]: pgmap v641: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-06-02T15:48:41.123 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:41.123 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:41.152 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:41.153 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:41.153 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 91 -lt 100 ']' 2024-06-02T15:48:41.154 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 91 + 1 2024-06-02T15:48:41.294 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=92 2024-06-02T15:48:41.294 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:41.406 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:41.406 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:41.410 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:41.412 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:41.412 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 -- 172.21.15.90:0/3260286951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc101440 msgr2=0x7f2ffc101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 --2- 172.21.15.90:0/3260286951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc101440 0x7f2ffc101860 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009900 tx=0x7f2fec02a400 comp rx=0 tx=0).stop 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 -- 172.21.15.90:0/3260286951 shutdown_connections 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 --2- 172.21.15.90:0/3260286951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc102ca0 0x7f2ffc109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 --2- 172.21.15.90:0/3260286951 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 --2- 172.21.15.90:0/3260286951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc101440 0x7f2ffc101860 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.557+0000 7f3002a00640 1 -- 172.21.15.90:0/3260286951 >> 172.21.15.90:0/3260286951 conn(0x7f2ffc0fc270 msgr2=0x7f2ffc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.558+0000 7f3002a00640 1 -- 172.21.15.90:0/3260286951 shutdown_connections 2024-06-02T15:48:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.558+0000 7f3002a00640 1 -- 172.21.15.90:0/3260286951 wait complete. 2024-06-02T15:48:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.619+0000 7f3002a00640 1 Processor -- start 2024-06-02T15:48:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.619+0000 7f3002a00640 1 -- start start 2024-06-02T15:48:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.620+0000 7f3002a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.620+0000 7f3002a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.620+0000 7f3002a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc102ca0 0x7f2ffc1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.620+0000 7f3002a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10e630 con 0x7f2ffc1022e0 2024-06-02T15:48:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.620+0000 7f3002a00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10e4b0 con 0x7f2ffc101440 2024-06-02T15:48:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f3002a00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10e7b0 con 0x7f2ffc102ca0 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:38116/0 (socket says 172.21.15.90:38116) 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3847864523 learned_addr learned my addr 172.21.15.90:0/3847864523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f3000a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc102ca0 0x7f2ffc1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3847864523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc102ca0 msgr2=0x7f2ffc1a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc102ca0 0x7f2ffc1a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc1022e0 msgr2=0x7f2ffc19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.621+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3847864523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1a2a90 con 0x7f2ffc101440 2024-06-02T15:48:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.622+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fec031500 tx=0x7f2fec003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.624+0000 7f2ff9600640 1 -- 172.21.15.90:0/3847864523 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec03f070 con 0x7f2ffc101440 2024-06-02T15:48:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.624+0000 7f2ff9600640 1 -- 172.21.15.90:0/3847864523 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec031c50 con 0x7f2ffc101440 2024-06-02T15:48:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.625+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc1a2d20 con 0x7f2ffc101440 2024-06-02T15:48:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.625+0000 7f2ff9600640 1 -- 172.21.15.90:0/3847864523 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec035030 con 0x7f2ffc101440 2024-06-02T15:48:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.628+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a31c0 con 0x7f2ffc101440 2024-06-02T15:48:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.629+0000 7f2ff9600640 1 -- 172.21.15.90:0/3847864523 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2fec042070 con 0x7f2ffc101440 2024-06-02T15:48:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.631+0000 7f2ff9600640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 0x7f2fe00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.632+0000 7f2ff9600640 1 -- 172.21.15.90:0/3847864523 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2fec106060 con 0x7f2ffc101440 2024-06-02T15:48:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.632+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 0x7f2fe00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.632+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 0x7f2fe00b4eb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc19c380 tx=0x7f2fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:42.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:42 smithi090 ceph-mon[31595]: pgmap v642: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:48:42.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:42 smithi120 ceph-mon[36451]: pgmap v642: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:48:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:42 smithi135 ceph-mon[36839]: pgmap v642: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:48:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.972+0000 7f30059e7740 1 Processor -- start 2024-06-02T15:48:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.973+0000 7f30059e7740 1 -- start start 2024-06-02T15:48:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.973+0000 7f30059e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f30059e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f30059e7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 0x5609a9dd0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f30059e7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5609a9d38b90 con 0x5609a9dce690 2024-06-02T15:48:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f30059e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609a9d28730 con 0x5609a9dc9520 2024-06-02T15:48:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f30059e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609a9d2c770 con 0x5609a9dc8b30 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f3000a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 0x5609a9dd0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:38122/0 (socket says 172.21.15.90:38122) 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53214/0 (socket says 172.21.15.90:53214) 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.974+0000 7f2ff3400640 1 -- 172.21.15.90:0/3528569637 learned_addr learned my addr 172.21.15.90:0/3528569637 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3528569637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 msgr2=0x5609a9dce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3528569637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 msgr2=0x5609a9dd0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 0x5609a9dd0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3528569637 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a9cc18b0 con 0x5609a9dc8b30 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f3000a00640 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 0x5609a9dd0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1a29e0 tx=0x7f2fec005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab35e856138f650 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.975+0000 7f2ff1200640 1 -- 172.21.15.90:0/3528569637 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0c91e0 con 0x5609a9dc8b30 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.976+0000 7f2ff1200640 1 -- 172.21.15.90:0/3528569637 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec105020 con 0x5609a9dc8b30 2024-06-02T15:48:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.976+0000 7f2ff1200640 1 -- 172.21.15.90:0/3528569637 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec03b460 con 0x5609a9dc8b30 2024-06-02T15:48:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.977+0000 7f30059e7740 1 -- 172.21.15.90:0/3528569637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 msgr2=0x5609a9dc8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.977+0000 7f30059e7740 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc1a29e0 tx=0x7f2fec005d50 comp rx=0 tx=0).stop 2024-06-02T15:48:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 -- 172.21.15.90:0/3528569637 shutdown_connections 2024-06-02T15:48:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dce690 0x5609a9dd0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dc9520 0x5609a9dce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 --2- 172.21.15.90:0/3528569637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dc8f50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 -- 172.21.15.90:0/3528569637 >> 172.21.15.90:0/3528569637 conn(0x5609a9dc41a0 msgr2=0x5609a9dc45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 -- 172.21.15.90:0/3528569637 shutdown_connections 2024-06-02T15:48:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:42.978+0000 7f30059e7740 1 -- 172.21.15.90:0/3528569637 wait complete. 2024-06-02T15:48:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.012+0000 7f30059e7740 1 -- start start 2024-06-02T15:48:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.055+0000 7f30059e7740 1 Processor -- start 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.056+0000 7f30059e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.056+0000 7f30059e7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.056+0000 7f30059e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.056+0000 7f30059e7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5609a9a0e790 con 0x5609a9dc9520 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f30059e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5609a9cd0bc0 con 0x5609a9dce690 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f30059e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609a9d2c770 con 0x5609a9dc8b30 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49216/0 (socket says 172.21.15.90:49216) 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53220/0 (socket says 172.21.15.90:53220) 2024-06-02T15:48:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 -- 172.21.15.90:0/1926170895 learned_addr learned my addr 172.21.15.90:0/1926170895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f3000a00640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 -- 172.21.15.90:0/1926170895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 msgr2=0x5609a9dd2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 -- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 msgr2=0x5609a9dd9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.057+0000 7f2ff3400640 1 -- 172.21.15.90:0/1926170895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a9a3d210 con 0x5609a9dc9520 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.058+0000 7f2ff3400640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003850 tx=0x7f2fe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7608d80f251fe2a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.058+0000 7f2fe9600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.058+0000 7f2fe9600640 1 -- 172.21.15.90:0/1926170895 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400e720 con 0x5609a9dc9520 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.058+0000 7f2fe9600640 1 -- 172.21.15.90:0/1926170895 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2fe400e8c0 con 0x5609a9dc9520 2024-06-02T15:48:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.059+0000 7f30059e7740 10 client.0 _ll_register_callbacks cb 0x5609a9cae400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.061+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fec105020 con 0x5609a9dc9520 2024-06-02T15:48:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.063+0000 7f2fe9600640 1 -- 172.21.15.90:0/1926170895 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fe400eac0 con 0x5609a9dc9520 2024-06-02T15:48:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.064+0000 7f30059e7740 10 client.15603 fetch_fsmap learned FSMap version 29 2024-06-02T15:48:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.064+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5609a9b09620 con 0x5609a9dc9520 2024-06-02T15:48:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.103+0000 7f2fe9600640 1 -- 172.21.15.90:0/1926170895 <== mon.0 v2:172.21.15.90:3300/0 4 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2fe400ec60 con 0x5609a9dc9520 2024-06-02T15:48:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.103+0000 7f30059e7740 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T15:48:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.121+0000 7f30059e7740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.121+0000 7f30059e7740 20 client.15603 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.121+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9dddf20 0x5609a9de03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.122+0000 7f30059e7740 4 client.15603 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:48:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.122+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a9d3aae0 con 0x5609a9dddf20 2024-06-02T15:48:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.122+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9dddf20 0x5609a9de03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.123+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9dddf20 0x5609a9de03e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73363c2d0e7a2a6e server_cookie=c5d27083de6b2bea in_seq=0 out_seq=0 2024-06-02T15:48:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.123+0000 7f2fe9600640 10 client.15603 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.124+0000 7f2fe9600640 1 -- 172.21.15.90:0/1926170895 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609a9d3aae0 con 0x5609a9dddf20 2024-06-02T15:48:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.124+0000 7f2fe9600640 10 client.15603 handle_command_reply: tid=0 2024-06-02T15:48:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 client.15603 shutdown 2024-06-02T15:48:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 msgr2=0x5609a9dd2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:43.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003850 tx=0x7f2fe4004730 comp rx=0 tx=0).stop 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 shutdown_connections 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9dddf20 0x5609a9de03e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc9520 0x5609a9dd2ba0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f2fe9600640 0 client.15603 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 --2- 172.21.15.90:0/1926170895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.125+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 >> 172.21.15.90:0/1926170895 conn(0x5609a9dc41a0 msgr2=0x5609a9dc3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.126+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 shutdown_connections 2024-06-02T15:48:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.126+0000 7f30059e7740 1 -- 172.21.15.90:0/1926170895 wait complete. 2024-06-02T15:48:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.126+0000 7f30059e7740 20 client.15603 trim_cache size 0 max 16384 2024-06-02T15:48:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 Processor -- start 2024-06-02T15:48:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 -- start start 2024-06-02T15:48:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dc77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.173+0000 7f30059e7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0120 con 0x5609a9dc8b30 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f30059e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ad9c0 con 0x5609a9dce690 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f30059e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ab810 con 0x5609a9dc9520 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f3000a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dc77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49220/0 (socket says 172.21.15.90:49220) 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f2ffbe00640 1 -- 172.21.15.90:0/3070758308 learned_addr learned my addr 172.21.15.90:0/3070758308 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.174+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 -- 172.21.15.90:0/3070758308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 msgr2=0x5609a9dc77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dc77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 -- 172.21.15.90:0/3070758308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 msgr2=0x5609a9dd2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 -- 172.21.15.90:0/3070758308 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a9dc8770 con 0x5609a9dc9520 2024-06-02T15:48:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:48:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2ff3400640 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd9ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009730 tx=0x7f2fe4022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1328f618ffc64388 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2febe00640 1 -- 172.21.15.90:0/3070758308 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40044c0 con 0x5609a9dc9520 2024-06-02T15:48:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2febe00640 1 -- 172.21.15.90:0/3070758308 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec105020 con 0x5609a9dc9520 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.175+0000 7f2febe00640 1 -- 172.21.15.90:0/3070758308 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400e810 con 0x5609a9dc9520 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 -- 172.21.15.90:0/3070758308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 msgr2=0x5609a9dd9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd9ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009730 tx=0x7f2fe4022000 comp rx=0 tx=0).stop 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 -- 172.21.15.90:0/3070758308 shutdown_connections 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dc77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd9ba0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 --2- 172.21.15.90:0/3070758308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 -- 172.21.15.90:0/3070758308 >> 172.21.15.90:0/3070758308 conn(0x5609a9dc41a0 msgr2=0x5609a9dc70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 -- 172.21.15.90:0/3070758308 shutdown_connections 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.177+0000 7f30059e7740 1 -- 172.21.15.90:0/3070758308 wait complete. 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.178+0000 7f30059e7740 1 -- start start 2024-06-02T15:48:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.179+0000 7f30059e7740 1 Processor -- start 2024-06-02T15:48:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.179+0000 7f30059e7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.179+0000 7f30059e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.179+0000 7f30059e7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f30059e7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1abe80 con 0x5609a9dc8b30 2024-06-02T15:48:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f30059e7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ad2f0 con 0x5609a9dce690 2024-06-02T15:48:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f30059e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ac550 con 0x5609a9dc9520 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f30059e7740 10 client.0 _ll_register_callbacks cb 0x7f2fe40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49226/0 (socket says 172.21.15.90:49226) 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ffbe00640 1 -- 172.21.15.90:0/2932035124 learned_addr learned my addr 172.21.15.90:0/2932035124 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f3000a00640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 -- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 msgr2=0x5609a9dd8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 -- 172.21.15.90:0/2932035124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 msgr2=0x5609a9dd70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ff3400640 1 -- 172.21.15.90:0/2932035124 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a9dbf440 con 0x5609a9dc9520 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.180+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.181+0000 7f2ff3400640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd75f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400efd0 tx=0x7f2fe40314f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5cf765657c386d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:48:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.181+0000 7f2fe9600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:48:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.181+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4031a30 con 0x5609a9dc9520 2024-06-02T15:48:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.181+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fec105020 con 0x5609a9dc9520 2024-06-02T15:48:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.181+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2fe4031bd0 con 0x5609a9dc9520 2024-06-02T15:48:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fe4031e10 con 0x5609a9dc9520 2024-06-02T15:48:43.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f30059e7740 10 client.25717 fetch_fsmap learned FSMap version 29 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5609a9dd9d30 con 0x5609a9dc9520 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec105020 con 0x5609a9dc9520 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2fe403b410 con 0x5609a9dc9520 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f30059e7740 10 client.25717 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f30059e7740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.182+0000 7f30059e7740 20 client.25717 populate_metadata read hostname 'smithi090' 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.183+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9e49150 0x5609a9e4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.183+0000 7f30059e7740 4 client.25717 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.183+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609a9d342c0 con 0x5609a9e49150 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.183+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9e49150 0x5609a9e4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.183+0000 7f2ffbe00640 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9e49150 0x5609a9e4b570 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40984cdd1b5c97bd server_cookie=8a0a4c16abebdb24 in_seq=0 out_seq=0 2024-06-02T15:48:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.184+0000 7f2fe9600640 10 client.25717 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.200+0000 7f2fe9600640 1 -- 172.21.15.90:0/2932035124 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5609a9d342c0 con 0x5609a9e49150 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.200+0000 7f2fe9600640 10 client.25717 handle_command_reply: tid=0 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.200+0000 7f30059e7740 1 client.25717 shutdown 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.200+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 msgr2=0x5609a9dd75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.200+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd75f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400efd0 tx=0x7f2fe40314f0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 shutdown_connections 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5609a9e49150 0x5609a9e4b570 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5609a9dce690 0x5609a9dd8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609a9dc9520 0x5609a9dd75f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f2fe9600640 0 client.25717 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 --2- 172.21.15.90:0/2932035124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609a9dc8b30 0x5609a9dd70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 >> 172.21.15.90:0/2932035124 conn(0x5609a9dc41a0 msgr2=0x5609a9dc9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 shutdown_connections 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 1 -- 172.21.15.90:0/2932035124 wait complete. 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.201+0000 7f30059e7740 20 client.25717 trim_cache size 0 max 16384 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:48:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:48:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 msgr2=0x7f2fe00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 0x7f2fe00b4eb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc19c380 tx=0x7f2fe4005f90 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 msgr2=0x7f2ffc19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fec031500 tx=0x7f2fec003c10 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 shutdown_connections 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2fe00b29f0 0x7f2fe00b4eb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ffc102ca0 0x7f2ffc1a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ffc1022e0 0x7f2ffc19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 --2- 172.21.15.90:0/3847864523 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2ffc101440 0x7f2ffc19ae70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.202+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 >> 172.21.15.90:0/3847864523 conn(0x7f2ffc0fc270 msgr2=0x7f2ffc10ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:48:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.203+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 shutdown_connections 2024-06-02T15:48:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:48:43.203+0000 7f3002a00640 1 -- 172.21.15.90:0/3847864523 wait complete. 2024-06-02T15:48:43.265 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:43.265 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:43.593 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:43.593 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:43.635 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:43.635 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:43.635 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 92 -lt 100 ']' 2024-06-02T15:48:43.637 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 92 + 1 2024-06-02T15:48:43.767 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=93 2024-06-02T15:48:43.768 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:43.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:43 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:43.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:43 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:43.835 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:43.835 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:43.839 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:43.840 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:43.841 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:43.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:43 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:44.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:44 smithi120 ceph-mon[36451]: pgmap v643: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:48:44.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:44 smithi135 ceph-mon[36839]: pgmap v643: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:48:44.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:44 smithi090 ceph-mon[31595]: pgmap v643: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 9 op/s 2024-06-02T15:48:45.348 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:45.348 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:45 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:45 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:45.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:45 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:48:45.848 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:45.848 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:45.888 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:45.888 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:45.888 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 93 -lt 100 ']' 2024-06-02T15:48:45.889 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 93 + 1 2024-06-02T15:48:45.976 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=94 2024-06-02T15:48:45.976 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:46.052 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:46.052 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:46.056 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:46.058 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:46.058 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:46 smithi120 ceph-mon[36451]: pgmap v644: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-06-02T15:48:46.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:46 smithi135 ceph-mon[36839]: pgmap v644: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-06-02T15:48:46.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:46 smithi090 ceph-mon[31595]: pgmap v644: 129 pgs: 129 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-06-02T15:48:47.598 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:47.598 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:47.990 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:47.990 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:48.013 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:48.013 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:48.013 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 94 -lt 100 ']' 2024-06-02T15:48:48.014 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 94 + 1 2024-06-02T15:48:48.107 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=95 2024-06-02T15:48:48.107 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:48.186 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:48.187 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:48.190 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:48.192 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:48.192 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:49.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:49 smithi090 ceph-mon[31595]: pgmap v645: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:48:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:49 smithi120 ceph-mon[36451]: pgmap v645: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:48:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:49 smithi135 ceph-mon[36839]: pgmap v645: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-06-02T15:48:49.648 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:49.648 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:50.103 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:50.103 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:50.143 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:50.143 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:50.143 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 95 -lt 100 ']' 2024-06-02T15:48:50.144 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 95 + 1 2024-06-02T15:48:50.231 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=96 2024-06-02T15:48:50.231 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:50.302 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:50.303 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:50.307 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:50.308 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:50.308 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:51.257 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:48:51.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:51 smithi090 ceph-mon[31595]: pgmap v646: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-06-02T15:48:51.531 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:51.531 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:51 smithi120 ceph-mon[36451]: pgmap v646: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-06-02T15:48:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:51 smithi135 ceph-mon[36839]: pgmap v646: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-06-02T15:48:51.862 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:51.862 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:51.913 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:51.914 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:51.914 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 96 -lt 100 ']' 2024-06-02T15:48:51.915 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 96 + 1 2024-06-02T15:48:52.042 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=97 2024-06-02T15:48:52.042 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:52.110 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:52.110 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:52.114 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:52.115 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:52.116 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:53 smithi090 ceph-mon[31595]: pgmap v647: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-02T15:48:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:53 smithi120 ceph-mon[36451]: pgmap v647: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-02T15:48:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:53 smithi135 ceph-mon[36839]: pgmap v647: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-06-02T15:48:53.681 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:53.681 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:54.256 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:54.256 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:54.306 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:54.307 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:54.307 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 97 -lt 100 ']' 2024-06-02T15:48:54.307 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 97 + 1 2024-06-02T15:48:54.384 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=98 2024-06-02T15:48:54.385 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:54.460 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:54.460 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:54.465 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:54.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:54.466 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:55.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:55 smithi090 ceph-mon[31595]: pgmap v648: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:48:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:55 smithi120 ceph-mon[36451]: pgmap v648: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:48:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:55 smithi135 ceph-mon[36839]: pgmap v648: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-06-02T15:48:55.923 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:55.923 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:56.328 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:56.328 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:56.415 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:56.415 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:56.415 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 98 -lt 100 ']' 2024-06-02T15:48:56.416 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 98 + 1 2024-06-02T15:48:56.534 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=99 2024-06-02T15:48:56.534 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:56.611 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:56.611 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:56.615 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:56.616 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:56.617 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:57 smithi120 ceph-mon[36451]: pgmap v649: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:48:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:57 smithi135 ceph-mon[36839]: pgmap v649: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:48:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:57 smithi090 ceph-mon[31595]: pgmap v649: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-06-02T15:48:58.298 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:58.298 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:58.739 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:48:58.739 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:48:58.755 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:58.755 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:48:58.755 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 99 -lt 100 ']' 2024-06-02T15:48:58.757 INFO:tasks.workunit.client.0.smithi090.stderr:++ expr 99 + 1 2024-06-02T15:48:58.886 INFO:tasks.workunit.client.0.smithi090.stderr:+ c=100 2024-06-02T15:48:58.887 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:48:58.977 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:48:58.977 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:48:58.981 INFO:tasks.workunit.client.0.smithi090.stderr:+ c1=2 2024-06-02T15:48:58.982 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo 3 2024-06-02T15:48:58.983 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-06-02T15:48:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:48:59 smithi120 ceph-mon[36451]: pgmap v650: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:48:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:48:59 smithi135 ceph-mon[36839]: pgmap v650: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:48:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:48:59 smithi090 ceph-mon[31595]: pgmap v650: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:49:00.639 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfacl d1 2024-06-02T15:49:00.639 INFO:tasks.workunit.client.0.smithi090.stderr:++ grep -c nobody:rw 2024-06-02T15:49:01.185 INFO:tasks.workunit.client.0.smithi090.stderr:+ c2=2 2024-06-02T15:49:01.185 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:49:01.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:49:01.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 2 -ne 2 ']' 2024-06-02T15:49:01.221 INFO:tasks.workunit.client.0.smithi090.stderr:+ '[' 100 -lt 100 ']' 2024-06-02T15:49:01.283 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir d1 2024-06-02T15:49:01.416 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2024-06-02T15:49:01.464 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2024-06-02T15:49:01.468 INFO:tasks.workunit.client.0.smithi090.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2024-06-02T15:49:01.468 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n system.posix_acl_access d1 2024-06-02T15:49:01.476 INFO:tasks.workunit.client.0.smithi090.stderr:d1: system.posix_acl_access: No such attribute 2024-06-02T15:49:01.476 INFO:tasks.workunit.client.0.smithi090.stderr:+ return 0 2024-06-02T15:49:01.477 INFO:tasks.workunit.client.0.smithi090.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2024-06-02T15:49:01.477 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n system.posix_acl_default . 2024-06-02T15:49:01.479 INFO:tasks.workunit.client.0.smithi090.stderr:.: system.posix_acl_default: No such attribute 2024-06-02T15:49:01.480 INFO:tasks.workunit.client.0.smithi090.stderr:+ return 0 2024-06-02T15:49:01.480 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir d1 2024-06-02T15:49:01.485 INFO:tasks.workunit.client.0.smithi090.stderr:+ cd .. 2024-06-02T15:49:01.485 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir testdir 2024-06-02T15:49:01.497 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo OK 2024-06-02T15:49:01.497 INFO:tasks.workunit.client.0.smithi090.stdout:OK 2024-06-02T15:49:01.499 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:49:01.499 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:49:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:01 smithi120 ceph-mon[36451]: pgmap v651: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:49:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:01 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:49:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:01 smithi135 ceph-mon[36839]: pgmap v651: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:49:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:01 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:49:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:01 smithi090 ceph-mon[31595]: pgmap v651: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.9 KiB/s wr, 8 op/s 2024-06-02T15:49:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:01 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:49:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 -- 172.21.15.90:0/2622326592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c103b00 msgr2=0x7fdd0c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 --2- 172.21.15.90:0/2622326592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c103b00 0x7fdd0c103f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009860 tx=0x7fdcf402a340 comp rx=0 tx=0).stop 2024-06-02T15:49:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 -- 172.21.15.90:0/2622326592 shutdown_connections 2024-06-02T15:49:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 --2- 172.21.15.90:0/2622326592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c1044c0 0x7fdd0c10ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 --2- 172.21.15.90:0/2622326592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c103b00 0x7fdd0c103f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.137+0000 7fdd11000640 1 --2- 172.21.15.90:0/2622326592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c102910 0x7fdd0c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.138+0000 7fdd11000640 1 -- 172.21.15.90:0/2622326592 >> 172.21.15.90:0/2622326592 conn(0x7fdd0c0fde80 msgr2=0x7fdd0c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.138+0000 7fdd11000640 1 -- 172.21.15.90:0/2622326592 shutdown_connections 2024-06-02T15:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.138+0000 7fdd11000640 1 -- 172.21.15.90:0/2622326592 wait complete. 2024-06-02T15:49:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.243+0000 7fdd11000640 1 Processor -- start 2024-06-02T15:49:03.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.244+0000 7fdd11000640 1 -- start start 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 0x7fdd0c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c1044c0 0x7fdd0c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10e440 con 0x7fdd0c1044c0 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10e2c0 con 0x7fdd0c103b00 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd11000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10e5c0 con 0x7fdd0c102910 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38400/0 (socket says 172.21.15.90:38400) 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd0b400640 1 -- 172.21.15.90:0/2013379304 learned_addr learned my addr 172.21.15.90:0/2013379304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.245+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 0x7fdd0c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0be00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c1044c0 0x7fdd0c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0b400640 1 -- 172.21.15.90:0/2013379304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 msgr2=0x7fdd0c199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 0x7fdd0c199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0b400640 1 -- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c1044c0 msgr2=0x7fdd0c1a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c1044c0 0x7fdd0c1a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd0b400640 1 -- 172.21.15.90:0/2013379304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1a0730 con 0x7fdd0c102910 2024-06-02T15:49:03.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.246+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 0x7fdd0c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.247+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9c0 tx=0x7fdcfc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.252+0000 7fdd08c00640 1 -- 172.21.15.90:0/2013379304 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00cde0 con 0x7fdd0c102910 2024-06-02T15:49:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.253+0000 7fdd08c00640 1 -- 172.21.15.90:0/2013379304 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcfc0108e0 con 0x7fdd0c102910 2024-06-02T15:49:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.253+0000 7fdd08c00640 1 -- 172.21.15.90:0/2013379304 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc018960 con 0x7fdd0c102910 2024-06-02T15:49:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.253+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1a0a20 con 0x7fdd0c102910 2024-06-02T15:49:03.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.253+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1a0ef0 con 0x7fdd0c102910 2024-06-02T15:49:03.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.258+0000 7fdd08c00640 1 -- 172.21.15.90:0/2013379304 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdcfc01f020 con 0x7fdd0c102910 2024-06-02T15:49:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.260+0000 7fdd08c00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 0x7fdcf00b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.260+0000 7fdd08c00640 1 -- 172.21.15.90:0/2013379304 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdcfc0e2840 con 0x7fdd0c102910 2024-06-02T15:49:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.261+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 0x7fdcf00b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.262+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 0x7fdcf00b4df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000c00 tx=0x7fdcf4030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.463+0000 7fdd155b7740 1 Processor -- start 2024-06-02T15:49:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.463+0000 7fdd155b7740 1 -- start start 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 0x55925f3e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55925f351b90 con 0x55925f3e26e0 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55925f341730 con 0x55925f3e1cf0 2024-06-02T15:49:03.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd155b7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55925f345770 con 0x55925f3e7850 2024-06-02T15:49:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.464+0000 7fdd03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39496/0 (socket says 172.21.15.90:39496) 2024-06-02T15:49:03.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd03e00640 1 -- 172.21.15.90:0/2423412698 learned_addr learned my addr 172.21.15.90:0/2423412698 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0be00640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 0x55925f3e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 -- 172.21.15.90:0/2423412698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 msgr2=0x55925f3e9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3e9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 -- 172.21.15.90:0/2423412698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 msgr2=0x55925f3e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 -- 172.21.15.90:0/2423412698 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925f2da8b0 con 0x55925f3e1cf0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.465+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 0x55925f3e2110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0058c0 tx=0x7fdcfc0aa150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ee6b160db78d7b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.466+0000 7fdd00c00640 1 -- 172.21.15.90:0/2423412698 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0a5050 con 0x55925f3e1cf0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.466+0000 7fdd00c00640 1 -- 172.21.15.90:0/2423412698 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcfc0b2070 con 0x55925f3e1cf0 2024-06-02T15:49:03.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.466+0000 7fdd00c00640 1 -- 172.21.15.90:0/2423412698 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0a5ce0 con 0x55925f3e1cf0 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.466+0000 7fdd155b7740 1 -- 172.21.15.90:0/2423412698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 msgr2=0x55925f3e2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.466+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 0x55925f3e2110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0058c0 tx=0x7fdcfc0aa150 comp rx=0 tx=0).stop 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 -- 172.21.15.90:0/2423412698 shutdown_connections 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3e9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e26e0 0x55925f3e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2423412698 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e1cf0 0x55925f3e2110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 -- 172.21.15.90:0/2423412698 >> 172.21.15.90:0/2423412698 conn(0x55925f3dd360 msgr2=0x55925f3dd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 -- 172.21.15.90:0/2423412698 shutdown_connections 2024-06-02T15:49:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.467+0000 7fdd155b7740 1 -- 172.21.15.90:0/2423412698 wait complete. 2024-06-02T15:49:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.502+0000 7fdd155b7740 1 -- start start 2024-06-02T15:49:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.512+0000 7fdd155b7740 1 Processor -- start 2024-06-02T15:49:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e7850 0x55925f3f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55925f027790 con 0x55925f3e7850 2024-06-02T15:49:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55925f2e9bc0 con 0x55925f3e26e0 2024-06-02T15:49:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd155b7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55925f3ea7a0 con 0x55925f3e1cf0 2024-06-02T15:49:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38420/0 (socket says 172.21.15.90:38420) 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd0b400640 1 -- 172.21.15.90:0/369724133 learned_addr learned my addr 172.21.15.90:0/369724133 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd0be00640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e7850 0x55925f3f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.514+0000 7fdd03e00640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 -- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 msgr2=0x55925f3ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 -- 172.21.15.90:0/369724133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e7850 msgr2=0x55925f3f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e7850 0x55925f3f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 -- 172.21.15.90:0/369724133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925f056210 con 0x55925f3e1cf0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd03e00640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd155b7740 10 client.0 _ll_register_callbacks cb 0x55925f2c7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdd0b400640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0a99e0 tx=0x7fdcfc0b2310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9118f195b6738e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdcf9600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:49:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdcf9600640 1 -- 172.21.15.90:0/369724133 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0c2070 con 0x55925f3e1cf0 2024-06-02T15:49:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.515+0000 7fdcf9600640 1 -- 172.21.15.90:0/369724133 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdcfc0b2420 con 0x55925f3e1cf0 2024-06-02T15:49:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.516+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcfc0b2070 con 0x55925f3e1cf0 2024-06-02T15:49:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.517+0000 7fdcf9600640 1 -- 172.21.15.90:0/369724133 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcfc0b2620 con 0x55925f3e1cf0 2024-06-02T15:49:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.517+0000 7fdd155b7740 10 client.25729 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.517+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55925f122620 con 0x55925f3e1cf0 2024-06-02T15:49:03.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:03 smithi090 ceph-mon[31595]: pgmap v652: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdcf9600640 1 -- 172.21.15.90:0/369724133 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdcfc010430 con 0x55925f3e1cf0 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 10 client.25729 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 20 client.25729 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f3f7060 0x55925f3f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 4 client.25729 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55925f353ae0 con 0x55925f3f7060 2024-06-02T15:49:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.550+0000 7fdd03e00640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f3f7060 0x55925f3f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.551+0000 7fdd03e00640 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f3f7060 0x55925f3f9450 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927f2010662a1815 server_cookie=7cbfc83b32e81be8 in_seq=0 out_seq=0 2024-06-02T15:49:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.551+0000 7fdcf9600640 10 client.25729 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.552+0000 7fdcf9600640 1 -- 172.21.15.90:0/369724133 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55925f353ae0 con 0x55925f3f7060 2024-06-02T15:49:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.552+0000 7fdcf9600640 10 client.25729 handle_command_reply: tid=0 2024-06-02T15:49:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 client.25729 shutdown 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 msgr2=0x55925f3ebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc0a99e0 tx=0x7fdcfc0b2310 comp rx=0 tx=0).stop 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 shutdown_connections 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f3f7060 0x55925f3f9450 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e7850 0x55925f3f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 --2- 172.21.15.90:0/369724133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e1cf0 0x55925f3ebb10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 >> 172.21.15.90:0/369724133 conn(0x55925f3dd360 msgr2=0x55925f3dc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.553+0000 7fdcf9600640 0 client.25729 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.554+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 shutdown_connections 2024-06-02T15:49:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.554+0000 7fdd155b7740 1 -- 172.21.15.90:0/369724133 wait complete. 2024-06-02T15:49:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.554+0000 7fdd155b7740 20 client.25729 trim_cache size 0 max 16384 2024-06-02T15:49:03.553 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2024-06-02T15:49:03.553 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2024-06-02T15:49:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:03 smithi120 ceph-mon[36451]: pgmap v652: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:49:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:03 smithi135 ceph-mon[36839]: pgmap v652: 129 pgs: 129 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 7 op/s 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 Processor -- start 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 -- start start 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 0x55925f3fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1add40 con 0x55925f3e1cf0 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1ab610 con 0x55925f3e7850 2024-06-02T15:49:03.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.578+0000 7fdd155b7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a9460 con 0x55925f3e26e0 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd03e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 0x55925f3fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36764/0 (socket says 172.21.15.90:36764) 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 -- 172.21.15.90:0/3468931396 learned_addr learned my addr 172.21.15.90:0/3468931396 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 -- 172.21.15.90:0/3468931396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 msgr2=0x55925f3fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 0x55925f3fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 -- 172.21.15.90:0/3468931396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 msgr2=0x55925f3fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd0be00640 1 -- 172.21.15.90:0/3468931396 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925f3e1930 con 0x55925f3e7850 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.579+0000 7fdd03e00640 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 0x55925f3fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.580+0000 7fdd0be00640 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400f530 tx=0x7fdd040095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48755c11000aa0d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.580+0000 7fdcfbe00640 1 -- 172.21.15.90:0/3468931396 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0401f070 con 0x55925f3e7850 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.580+0000 7fdcfbe00640 1 -- 172.21.15.90:0/3468931396 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd04004d20 con 0x55925f3e7850 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.580+0000 7fdcfbe00640 1 -- 172.21.15.90:0/3468931396 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04005670 con 0x55925f3e7850 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.581+0000 7fdd155b7740 1 -- 172.21.15.90:0/3468931396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 msgr2=0x55925f3fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.581+0000 7fdd155b7740 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400f530 tx=0x7fdd040095c0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 -- 172.21.15.90:0/3468931396 shutdown_connections 2024-06-02T15:49:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e7850 0x55925f3fe570 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e26e0 0x55925f3fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 --2- 172.21.15.90:0/3468931396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 -- 172.21.15.90:0/3468931396 >> 172.21.15.90:0/3468931396 conn(0x55925f3dd360 msgr2=0x55925f3e3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 -- 172.21.15.90:0/3468931396 shutdown_connections 2024-06-02T15:49:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.584+0000 7fdd155b7740 1 -- 172.21.15.90:0/3468931396 wait complete. 2024-06-02T15:49:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.585+0000 7fdd155b7740 1 -- start start 2024-06-02T15:49:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.585+0000 7fdd155b7740 1 Processor -- start 2024-06-02T15:49:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1a9ad0 con 0x55925f3e1cf0 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1aaf40 con 0x55925f3e26e0 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1aa1a0 con 0x55925f3e7850 2024-06-02T15:49:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd155b7740 10 client.0 _ll_register_callbacks cb 0x7fdcfc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd0b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39516/0 (socket says 172.21.15.90:39516) 2024-06-02T15:49:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd0b400640 1 -- 172.21.15.90:0/2318815337 learned_addr learned my addr 172.21.15.90:0/2318815337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd0be00640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.586+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 -- 172.21.15.90:0/2318815337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 msgr2=0x55925f3f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 -- 172.21.15.90:0/2318815337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 msgr2=0x55925f3f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 -- 172.21.15.90:0/2318815337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925f3dd190 con 0x55925f3e26e0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdd03e00640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3f2e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55925f3e2ae0 tx=0x7fdcf4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca8ce32ea528273 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdcf9600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.587+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf4037040 con 0x55925f3e26e0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd04004d20 con 0x55925f3e26e0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fdcf4050e00 con 0x55925f3e26e0 2024-06-02T15:49:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcf4060050 con 0x55925f3e26e0 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 10 client.25772 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55925f46aa60 con 0x55925f3e26e0 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd04004d20 con 0x55925f3e26e0 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdcf4003b20 con 0x55925f3e26e0 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 10 client.25772 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 10 client.25772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 20 client.25772 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.588+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f46b3c0 0x55925f46d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:03.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.589+0000 7fdd155b7740 4 client.25772 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:49:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.589+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55925f34d2c0 con 0x55925f46b3c0 2024-06-02T15:49:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.589+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f46b3c0 0x55925f46d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.589+0000 7fdd0b400640 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f46b3c0 0x55925f46d7e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5253b93823fc8d3b server_cookie=7fdc581d36ca5c6d in_seq=0 out_seq=0 2024-06-02T15:49:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.589+0000 7fdcf9600640 10 client.25772 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdcf9600640 1 -- 172.21.15.90:0/2318815337 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55925f34d2c0 con 0x55925f46b3c0 2024-06-02T15:49:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdcf9600640 10 client.25772 handle_command_reply: tid=0 2024-06-02T15:49:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 client.25772 shutdown 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 msgr2=0x55925f3f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3f2e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55925f3e2ae0 tx=0x7fdcf4005f90 comp rx=0 tx=0).stop 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 shutdown_connections 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55925f46b3c0 0x55925f46d7e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55925f3e7850 0x55925f3f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55925f3e26e0 0x55925f3f2e80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 --2- 172.21.15.90:0/2318815337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55925f3e1cf0 0x55925f3f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 >> 172.21.15.90:0/2318815337 conn(0x55925f3dd360 msgr2=0x55925f3dbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdcf9600640 0 client.25772 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 shutdown_connections 2024-06-02T15:49:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.608+0000 7fdd155b7740 1 -- 172.21.15.90:0/2318815337 wait complete. 2024-06-02T15:49:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.609+0000 7fdd155b7740 20 client.25772 trim_cache size 0 max 16384 2024-06-02T15:49:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:49:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:49:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.609+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 msgr2=0x7fdcf00b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.609+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 0x7fdcf00b4df0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4000c00 tx=0x7fdcf4030470 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.609+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 msgr2=0x7fdd0c198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.609+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9c0 tx=0x7fdcfc00ee80 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 shutdown_connections 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdcf00b2930 0x7fdcf00b4df0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd0c1044c0 0x7fdd0c1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd0c103b00 0x7fdd0c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 --2- 172.21.15.90:0/2013379304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd0c102910 0x7fdd0c198b10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 >> 172.21.15.90:0/2013379304 conn(0x7fdd0c0fde80 msgr2=0x7fdd0c106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 shutdown_connections 2024-06-02T15:49:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:03.610+0000 7fdd11000640 1 -- 172.21.15.90:0/2013379304 wait complete. 2024-06-02T15:49:03.745 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:49:03.745 INFO:tasks.workunit.client.0.smithi090.stderr:+ file=test_chmod.97149 2024-06-02T15:49:03.745 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo foo 2024-06-02T15:49:03.756 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:03.756 INFO:tasks.workunit.client.0.smithi090.stderr:+ chmod 400 test_chmod.97149 2024-06-02T15:49:03.775 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:03.775 INFO:tasks.workunit.client.0.smithi090.stderr:+ check_perms test_chmod.97149 -r-------- 2024-06-02T15:49:03.775 INFO:tasks.workunit.client.0.smithi090.stderr:+ file=test_chmod.97149 2024-06-02T15:49:03.776 INFO:tasks.workunit.client.0.smithi090.stderr:++ ls -la test_chmod.97149 2024-06-02T15:49:03.823 INFO:tasks.workunit.client.0.smithi090.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Jun 2 15:49 test_chmod.97149' 2024-06-02T15:49:03.823 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:03.823 INFO:tasks.workunit.client.0.smithi090.stderr:+ perms=-r-------- 2024-06-02T15:49:03.826 INFO:tasks.workunit.client.0.smithi090.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Jun 2 15:49 test_chmod.97149 2024-06-02T15:49:03.827 INFO:tasks.workunit.client.0.smithi090.stderr:++ awk '{print $1}' 2024-06-02T15:49:04.201 INFO:tasks.workunit.client.0.smithi090.stderr:+ test -r-------- '!=' -r--------. 2024-06-02T15:49:04.202 INFO:tasks.workunit.client.0.smithi090.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Jun 2 15:49 test_chmod.97149 2024-06-02T15:49:04.204 INFO:tasks.workunit.client.0.smithi090.stderr:++ awk '{print $1}' 2024-06-02T15:49:04.210 INFO:tasks.workunit.client.0.smithi090.stderr:+ test -r--------. '!=' -r--------. 2024-06-02T15:49:04.210 INFO:tasks.workunit.client.0.smithi090.stderr:+ set +e 2024-06-02T15:49:04.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo bar 2024-06-02T15:49:04.211 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.97149: Permission denied 2024-06-02T15:49:04.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 1 = 0 2024-06-02T15:49:04.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:49:04.211 INFO:tasks.workunit.client.0.smithi090.stderr:+ chmod 600 test_chmod.97149 2024-06-02T15:49:04.213 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo bar 2024-06-02T15:49:04.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:04.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ check_perms test_chmod.97149 -rw------- 2024-06-02T15:49:04.214 INFO:tasks.workunit.client.0.smithi090.stderr:+ file=test_chmod.97149 2024-06-02T15:49:04.215 INFO:tasks.workunit.client.0.smithi090.stderr:++ ls -la test_chmod.97149 2024-06-02T15:49:04.220 INFO:tasks.workunit.client.0.smithi090.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Jun 2 15:49 test_chmod.97149' 2024-06-02T15:49:04.220 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:04.220 INFO:tasks.workunit.client.0.smithi090.stderr:+ perms=-rw------- 2024-06-02T15:49:04.222 INFO:tasks.workunit.client.0.smithi090.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Jun 2 15:49 test_chmod.97149 2024-06-02T15:49:04.222 INFO:tasks.workunit.client.0.smithi090.stderr:++ awk '{print $1}' 2024-06-02T15:49:04.227 INFO:tasks.workunit.client.0.smithi090.stderr:+ test -rw------- '!=' -rw-------. 2024-06-02T15:49:04.229 INFO:tasks.workunit.client.0.smithi090.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Jun 2 15:49 test_chmod.97149 2024-06-02T15:49:04.229 INFO:tasks.workunit.client.0.smithi090.stderr:++ awk '{print $1}' 2024-06-02T15:49:04.234 INFO:tasks.workunit.client.0.smithi090.stderr:+ test -rw-------. '!=' -rw-------. 2024-06-02T15:49:04.235 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo foo 2024-06-02T15:49:04.235 INFO:tasks.workunit.client.0.smithi090.stderr:+ test 0 '!=' 0 2024-06-02T15:49:04.236 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:49:04.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:49:04.375 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2024-06-02T15:49:04.376 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2024-06-02T15:49:04.469 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:49:04.469 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p testdir 2024-06-02T15:49:04.474 INFO:tasks.workunit.client.0.smithi090.stderr:+ file=test_chmod.97205 2024-06-02T15:49:04.474 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo foo 2024-06-02T15:49:04.476 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo chmod 600 testdir 2024-06-02T15:49:04.526 INFO:tasks.workunit.client.0.smithi090.stderr:+ expect_failure cat testdir/test_chmod.97205 2024-06-02T15:49:04.526 INFO:tasks.workunit.client.0.smithi090.stderr:+ cat testdir/test_chmod.97205 2024-06-02T15:49:04.530 INFO:tasks.workunit.client.0.smithi090.stderr:cat: testdir/test_chmod.97205: Permission denied 2024-06-02T15:49:04.530 INFO:tasks.workunit.client.0.smithi090.stderr:+ return 0 2024-06-02T15:49:04.530 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo cat testdir/test_chmod.97205 2024-06-02T15:49:04.557 INFO:tasks.workunit.client.0.smithi090.stdout:foo 2024-06-02T15:49:04.561 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:49:04.562 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:49:04.656 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2024-06-02T15:49:04.656 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2024-06-02T15:49:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:05 smithi120 ceph-mon[36451]: pgmap v653: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:49:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:05 smithi135 ceph-mon[36839]: pgmap v653: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:49:05.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:05 smithi090 ceph-mon[31595]: pgmap v653: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-06-02T15:49:05.630 INFO:tasks.workunit.client.0.smithi090.stdout:ok 2024-06-02T15:49:05.637 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:49:05.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:49:05.720 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2024-06-02T15:49:05.720 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2024-06-02T15:49:05.815 INFO:tasks.workunit.client.0.smithi090.stdout:creating folder hierarchy 2024-06-02T15:49:07.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:07 smithi090 ceph-mon[31595]: pgmap v654: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-06-02T15:49:07.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:49:07.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:49:07.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:07 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:49:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:07 smithi120 ceph-mon[36451]: pgmap v654: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-06-02T15:49:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:49:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:49:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:07 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:49:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:07 smithi135 ceph-mon[36839]: pgmap v654: 129 pgs: 129 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.9 KiB/s wr, 12 op/s 2024-06-02T15:49:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:49:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:49:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:07 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:49:08.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 -- 172.21.15.90:0/2761323595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010ff70 msgr2=0x7f4520112360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 --2- 172.21.15.90:0/2761323595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010ff70 0x7f4520112360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f451800b0a0 tx=0x7f451800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 -- 172.21.15.90:0/2761323595 shutdown_connections 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 --2- 172.21.15.90:0/2761323595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f452010ff70 0x7f4520112360 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 --2- 172.21.15.90:0/2761323595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520075770 0x7f4520075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 --2- 172.21.15.90:0/2761323595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520076090 0x7f452010fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 -- 172.21.15.90:0/2761323595 >> 172.21.15.90:0/2761323595 conn(0x7f452007a610 msgr2=0x7f452007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 -- 172.21.15.90:0/2761323595 shutdown_connections 2024-06-02T15:49:08.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.421+0000 7f4524a00700 1 -- 172.21.15.90:0/2761323595 wait complete. 2024-06-02T15:49:08.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.422+0000 7f4524a00700 1 Processor -- start 2024-06-02T15:49:08.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.422+0000 7f4524a00700 1 -- start start 2024-06-02T15:49:08.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 0x7f452019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010ff70 0x7f45201a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4520115970 con 0x7f452010ff70 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45201157f0 con 0x7f4520075770 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f4524a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4520115af0 con 0x7f4520076090 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f451de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f451de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36854/0 (socket says 172.21.15.90:36854) 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f451de00700 1 -- 172.21.15.90:0/2948632278 learned_addr learned my addr 172.21.15.90:0/2948632278 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f451d400700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 0x7f452019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.423+0000 7f451e800700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010ff70 0x7f45201a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 -- 172.21.15.90:0/2948632278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 msgr2=0x7f452019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 -- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010ff70 msgr2=0x7f45201a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010ff70 0x7f45201a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 -- 172.21.15.90:0/2948632278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201a0d90 con 0x7f4520076090 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451de00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f451d400700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 0x7f452019c2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9c0 tx=0x7f450c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.424+0000 7f4516a00700 1 -- 172.21.15.90:0/2948632278 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00cbd0 con 0x7f4520076090 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.425+0000 7f4516a00700 1 -- 172.21.15.90:0/2948632278 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f450c0044d0 con 0x7f4520076090 2024-06-02T15:49:08.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.425+0000 7f4516a00700 1 -- 172.21.15.90:0/2948632278 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c003c90 con 0x7f4520076090 2024-06-02T15:49:08.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.425+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a1080 con 0x7f4520076090 2024-06-02T15:49:08.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.425+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a8900 con 0x7f4520076090 2024-06-02T15:49:08.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.430+0000 7f4516a00700 1 -- 172.21.15.90:0/2948632278 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f450c003e50 con 0x7f4520076090 2024-06-02T15:49:08.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.432+0000 7f4516a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 0x7f45040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.432+0000 7f4516a00700 1 -- 172.21.15.90:0/2948632278 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f450c0e28a0 con 0x7f4520076090 2024-06-02T15:49:08.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.432+0000 7f451de00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 0x7f45040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.433+0000 7f451de00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 0x7f45040b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4510011590 tx=0x7f4510009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 Processor -- start 2024-06-02T15:49:08.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 -- start start 2024-06-02T15:49:08.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562f16c94e40 con 0x562f16d3beb0 2024-06-02T15:49:08.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f16c849e0 con 0x562f16d36880 2024-06-02T15:49:08.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.446+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f16c88a20 con 0x562f16d3c7d0 2024-06-02T15:49:08.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451d400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36866/0 (socket says 172.21.15.90:36866) 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 -- 172.21.15.90:0/1199357640 learned_addr learned my addr 172.21.15.90:0/1199357640 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451de00700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 -- 172.21.15.90:0/1199357640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 msgr2=0x562f16d36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 -- 172.21.15.90:0/1199357640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 msgr2=0x562f16d3c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451e800700 1 -- 172.21.15.90:0/1199357640 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f16c4d420 con 0x562f16d36880 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.447+0000 7f451d400700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.448+0000 7f451de00700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.448+0000 7f451e800700 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f452006ad10 tx=0x7f4518009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658221a86ed8a493 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.448+0000 7f450b400700 1 -- 172.21.15.90:0/1199357640 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451800bcb0 con 0x562f16d36880 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.448+0000 7f450b400700 1 -- 172.21.15.90:0/1199357640 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518009460 con 0x562f16d36880 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.448+0000 7f450b400700 1 -- 172.21.15.90:0/1199357640 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518007e20 con 0x562f16d36880 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 -- 172.21.15.90:0/1199357640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 msgr2=0x562f16d36ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f452006ad10 tx=0x7f4518009df0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 -- 172.21.15.90:0/1199357640 shutdown_connections 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d36ce0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/1199357640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 -- 172.21.15.90:0/1199357640 >> 172.21.15.90:0/1199357640 conn(0x562f16c64be0 msgr2=0x562f16d31010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.449+0000 7f452f2f6b80 1 -- 172.21.15.90:0/1199357640 shutdown_connections 2024-06-02T15:49:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.450+0000 7f452f2f6b80 1 -- 172.21.15.90:0/1199357640 wait complete. 2024-06-02T15:49:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.450+0000 7f452f2f6b80 1 -- start start 2024-06-02T15:49:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 Processor -- start 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562f16b8afa0 con 0x562f16d36880 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562f16c25760 con 0x562f16d3beb0 2024-06-02T15:49:08.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.451+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f16d431e0 con 0x562f16d3c7d0 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f452f2f6b80 10 client.0 _ll_register_callbacks cb 0x562f16c2ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39622/0 (socket says 172.21.15.90:39622) 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451d400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36870/0 (socket says 172.21.15.90:36870) 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451de00700 1 -- 172.21.15.90:0/2670759601 learned_addr learned my addr 172.21.15.90:0/2670759601 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.452+0000 7f451e800700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 -- 172.21.15.90:0/2670759601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 msgr2=0x562f16d3fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d3fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 -- 172.21.15.90:0/2670759601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 msgr2=0x562f16d3f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 -- 172.21.15.90:0/2670759601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f16bd48d0 con 0x562f16d36880 2024-06-02T15:49:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451d400700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f451de00700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f451001ed30 tx=0x7f45100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae47779ef2fbd91 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f44ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451001e260 con 0x562f16d36880 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.453+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f451001cb10 con 0x562f16d36880 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.454+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4518009460 con 0x562f16d36880 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.454+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f451001cd10 con 0x562f16d36880 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.454+0000 7f452f2f6b80 10 client.15657 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.454+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f16c55060 con 0x562f16d36880 2024-06-02T15:49:08.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.454+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518009460 con 0x562f16d36880 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4510025070 con 0x562f16d36880 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 10 client.15657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 20 client.15657 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16d4fe00 0x562f16d521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 4 client.15657 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.455+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f16c96d90 con 0x562f16d4fe00 2024-06-02T15:49:08.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.457+0000 7f451d400700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16d4fe00 0x562f16d521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.458+0000 7f451d400700 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16d4fe00 0x562f16d521f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22979aed141ab64a server_cookie=34be9c20c8febdf1 in_seq=0 out_seq=0 2024-06-02T15:49:08.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.458+0000 7f44ffe00700 10 client.15657 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:49:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.459+0000 7f44ffe00700 1 -- 172.21.15.90:0/2670759601 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562f16c96d90 con 0x562f16d4fe00 2024-06-02T15:49:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.459+0000 7f44ffe00700 10 client.15657 handle_command_reply: tid=0 2024-06-02T15:49:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.459+0000 7f452f2f6b80 1 client.15657 shutdown 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 msgr2=0x562f16d42530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f451001ed30 tx=0x7f45100047c0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 shutdown_connections 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16d4fe00 0x562f16d521f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3c7d0 0x562f16d3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d36880 0x562f16d42530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/2670759601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3beb0 0x562f16d3f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 >> 172.21.15.90:0/2670759601 conn(0x562f16c64be0 msgr2=0x562f16d35250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.460+0000 7f44ffe00700 0 client.15657 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:49:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.461+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 shutdown_connections 2024-06-02T15:49:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.461+0000 7f452f2f6b80 1 -- 172.21.15.90:0/2670759601 wait complete. 2024-06-02T15:49:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.461+0000 7f452f2f6b80 20 client.15657 trim_cache size 0 max 16384 2024-06-02T15:49:08.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 Processor -- start 2024-06-02T15:49:08.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 -- start start 2024-06-02T15:49:08.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d36880 0x562f16d36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b3e40 con 0x562f16d3beb0 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b16d0 con 0x562f16d3c7d0 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.483+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45201af520 con 0x562f16d36880 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36880/0 (socket says 172.21.15.90:36880) 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 -- 172.21.15.90:0/3762003783 learned_addr learned my addr 172.21.15.90:0/3762003783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451d400700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451de00700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d36880 0x562f16d36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 -- 172.21.15.90:0/3762003783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d36880 msgr2=0x562f16d36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d36880 0x562f16d36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 -- 172.21.15.90:0/3762003783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 msgr2=0x562f16d3e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451e800700 1 -- 172.21.15.90:0/3762003783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f16d4cbd0 con 0x562f16d3c7d0 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.484+0000 7f451d400700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.485+0000 7f451e800700 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f451800b0a0 tx=0x7f4518006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1827de27625bc2f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.485+0000 7f450aa00700 1 -- 172.21.15.90:0/3762003783 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451800bdf0 con 0x562f16d3c7d0 2024-06-02T15:49:08.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.485+0000 7f450aa00700 1 -- 172.21.15.90:0/3762003783 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45180074a0 con 0x562f16d3c7d0 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.485+0000 7f450aa00700 1 -- 172.21.15.90:0/3762003783 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518005280 con 0x562f16d3c7d0 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3762003783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 msgr2=0x562f16d3e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f451800b0a0 tx=0x7f4518006020 comp rx=0 tx=0).stop 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3762003783 shutdown_connections 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d3c7d0 0x562f16d3e720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d36880 0x562f16d36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3762003783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3beb0 0x562f16d3e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3762003783 >> 172.21.15.90:0/3762003783 conn(0x562f16c64be0 msgr2=0x562f16d2ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.486+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3762003783 shutdown_connections 2024-06-02T15:49:08.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.487+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3762003783 wait complete. 2024-06-02T15:49:08.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.487+0000 7f452f2f6b80 1 -- start start 2024-06-02T15:49:08.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.487+0000 7f452f2f6b80 1 Processor -- start 2024-06-02T15:49:08.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 0x562f16d57080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3c7d0 0x562f16d575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45201afb90 con 0x562f16d3c7d0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b1000 con 0x562f16d36880 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b0260 con 0x562f16d3beb0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.488+0000 7f452f2f6b80 10 client.0 _ll_register_callbacks cb 0x562f16c2ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36884/0 (socket says 172.21.15.90:36884) 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 -- 172.21.15.90:0/3189839995 learned_addr learned my addr 172.21.15.90:0/3189839995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451d400700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 0x562f16d57080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451e800700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3c7d0 0x562f16d575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 -- 172.21.15.90:0/3189839995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 msgr2=0x562f16d57080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 0x562f16d57080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 -- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3c7d0 msgr2=0x562f16d575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3c7d0 0x562f16d575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451de00700 1 -- 172.21.15.90:0/3189839995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f16d57b00 con 0x562f16d36880 2024-06-02T15:49:08.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.489+0000 7f451d400700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 0x562f16d57080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.490+0000 7f451de00700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4510011ce0 tx=0x7f4510005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ed05688a4bc8b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.490+0000 7f44ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.490+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510038020 con 0x562f16d36880 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.490+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4510034750 con 0x562f16d36880 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.490+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45180074a0 con 0x562f16d36880 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.491+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4510043050 con 0x562f16d36880 2024-06-02T15:49:08.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.491+0000 7f452f2f6b80 10 client.25808 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:08.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.491+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f16dc2ea0 con 0x562f16d36880 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45180074a0 con 0x562f16d36880 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f4510025040 con 0x562f16d36880 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 10 client.25808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 20 client.25808 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16dc35a0 0x562f16dc5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 4 client.25808 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f16c90570 con 0x562f16dc35a0 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.492+0000 7f451d400700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16dc35a0 0x562f16dc5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:08.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f451d400700 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16dc35a0 0x562f16dc5a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9186ba0b0227c8ff server_cookie=aa1d5a7e31d071f in_seq=0 out_seq=0 2024-06-02T15:49:08.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f44ffe00700 10 client.25808 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:49:08.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f44ffe00700 1 -- 172.21.15.90:0/3189839995 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f16c90570 con 0x562f16dc35a0 2024-06-02T15:49:08.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f44ffe00700 10 client.25808 handle_command_reply: tid=0 2024-06-02T15:49:08.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 client.25808 shutdown 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 msgr2=0x562f16d48850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4510011ce0 tx=0x7f4510005800 comp rx=0 tx=0).stop 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 shutdown_connections 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x562f16dc35a0 0x562f16dc5a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562f16d3c7d0 0x562f16d575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562f16d36880 0x562f16d48850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f44ffe00700 0 client.25808 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 --2- 172.21.15.90:0/3189839995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f16d3beb0 0x562f16d57080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.493+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 >> 172.21.15.90:0/3189839995 conn(0x562f16c64be0 msgr2=0x562f16d399c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.494+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 shutdown_connections 2024-06-02T15:49:08.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.494+0000 7f452f2f6b80 1 -- 172.21.15.90:0/3189839995 wait complete. 2024-06-02T15:49:08.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.494+0000 7f452f2f6b80 20 client.25808 trim_cache size 0 max 16384 2024-06-02T15:49:08.493 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:49:08.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 msgr2=0x7f45040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 0x7f45040b4dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4510011590 tx=0x7f4510009260 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 msgr2=0x7f452019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 0x7f452019c2c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f450c00e9c0 tx=0x7f450c00ecd0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 shutdown_connections 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45040b2900 0x7f45040b4dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010ff70 0x7f45201a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4520075770 0x7f452019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 --2- 172.21.15.90:0/2948632278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4520076090 0x7f452019c2c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 >> 172.21.15.90:0/2948632278 conn(0x7f452007a610 msgr2=0x7f4520100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 shutdown_connections 2024-06-02T15:49:08.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:08.495+0000 7f4524a00700 1 -- 172.21.15.90:0/2948632278 wait complete. 2024-06-02T15:49:09.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:09 smithi090 ceph-mon[31595]: pgmap v655: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-06-02T15:49:09.385 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:49:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:09 smithi120 ceph-mon[36451]: pgmap v655: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-06-02T15:49:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:09 smithi135 ceph-mon[36839]: pgmap v655: 129 pgs: 129 active+clean; 4.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 154 KiB/s wr, 16 op/s 2024-06-02T15:49:09.862 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:49:11.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:11 smithi090 ceph-mon[31595]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 335 KiB/s wr, 20 op/s 2024-06-02T15:49:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:11 smithi120 ceph-mon[36451]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 335 KiB/s wr, 20 op/s 2024-06-02T15:49:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:11 smithi135 ceph-mon[36839]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 335 KiB/s wr, 20 op/s 2024-06-02T15:49:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 -- 172.21.15.90:0/2738762892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 msgr2=0x7effe0105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 --2- 172.21.15.90:0/2738762892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe0105ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7effcc009990 tx=0x7effcc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 -- 172.21.15.90:0/2738762892 shutdown_connections 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 --2- 172.21.15.90:0/2738762892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe0106010 0x7effe0112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 --2- 172.21.15.90:0/2738762892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe010b650 0x7effe010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 --2- 172.21.15.90:0/2738762892 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe0105ad0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 -- 172.21.15.90:0/2738762892 >> 172.21.15.90:0/2738762892 conn(0x7effe0101120 msgr2=0x7effe0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 -- 172.21.15.90:0/2738762892 shutdown_connections 2024-06-02T15:49:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.952+0000 7effe5c00700 1 -- 172.21.15.90:0/2738762892 wait complete. 2024-06-02T15:49:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 Processor -- start 2024-06-02T15:49:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 -- start start 2024-06-02T15:49:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 0x7effe019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 0x7effe01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7effe01158e0 con 0x7effe010b650 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe0115760 con 0x7effe0105650 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effe5c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effe0115a60 con 0x7effe0106010 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effdfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.954+0000 7effdfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34798/0 (socket says 172.21.15.90:34798) 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 -- 172.21.15.90:0/4014522622 learned_addr learned my addr 172.21.15.90:0/4014522622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:11.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdf400700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 0x7effe019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 -- 172.21.15.90:0/4014522622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 msgr2=0x7effe019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effe5200700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 0x7effe01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 0x7effe019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 -- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 msgr2=0x7effe01a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 0x7effe01a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 -- 172.21.15.90:0/4014522622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe01a0d40 con 0x7effe0105650 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdf400700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 0x7effe019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:11.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effe5200700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 0x7effe01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.955+0000 7effdfe00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7effd000e9a0 tx=0x7effd000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.956+0000 7effdcc00700 1 -- 172.21.15.90:0/4014522622 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd000cbd0 con 0x7effe0105650 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.956+0000 7effdcc00700 1 -- 172.21.15.90:0/4014522622 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd000cd70 con 0x7effe0105650 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.956+0000 7effdcc00700 1 -- 172.21.15.90:0/4014522622 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0018680 con 0x7effe0105650 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.956+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe01a1030 con 0x7effe0105650 2024-06-02T15:49:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.956+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01a88b0 con 0x7effe0105650 2024-06-02T15:49:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.960+0000 7effdcc00700 1 -- 172.21.15.90:0/4014522622 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7effd0010430 con 0x7effe0105650 2024-06-02T15:49:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.961+0000 7efff1b25b80 1 Processor -- start 2024-06-02T15:49:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.961+0000 7efff1b25b80 1 -- start start 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86711ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f8670c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557f86669e40 con 0x557f867117c0 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f866599e0 con 0x557f867120e0 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7efff1b25b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f8665da20 con 0x557f8670c390 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7effdcc00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 0x7effc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7effdfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.962+0000 7effdfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34802/0 (socket says 172.21.15.90:34802) 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effdfe00700 1 -- 172.21.15.90:0/113614560 learned_addr learned my addr 172.21.15.90:0/113614560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effdcc00700 1 -- 172.21.15.90:0/4014522622 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7effd00e27c0 con 0x7effe0105650 2024-06-02T15:49:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86711ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 -- 172.21.15.90:0/113614560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 msgr2=0x557f8670c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effdf400700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f8670c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f8670c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 -- 172.21.15.90:0/113614560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 msgr2=0x557f8670bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effe5200700 1 -- 172.21.15.90:0/113614560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f86622420 con 0x557f867117c0 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.963+0000 7effdfe00700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effe5200700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 0x7effc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effdf400700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f8670c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effe5200700 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86711ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effd80094f0 tx=0x7effd80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3770f2d7d2eb769b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effe5200700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 0x7effc40b4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effd8020000 tx=0x7effd801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effd5600700 1 -- 172.21.15.90:0/113614560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8004bb0 con 0x557f867117c0 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effd5600700 1 -- 172.21.15.90:0/113614560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd8004d50 con 0x557f867117c0 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.964+0000 7effd5600700 1 -- 172.21.15.90:0/113614560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8017630 con 0x557f867117c0 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 -- 172.21.15.90:0/113614560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 msgr2=0x557f86711ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86711ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effd80094f0 tx=0x7effd80077e0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 -- 172.21.15.90:0/113614560 shutdown_connections 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f8670c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86711ba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 --2- 172.21.15.90:0/113614560 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f8670bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.965+0000 7efff1b25b80 1 -- 172.21.15.90:0/113614560 >> 172.21.15.90:0/113614560 conn(0x557f86639be0 msgr2=0x557f86706810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.967+0000 7efff1b25b80 1 -- 172.21.15.90:0/113614560 shutdown_connections 2024-06-02T15:49:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.967+0000 7efff1b25b80 1 -- 172.21.15.90:0/113614560 wait complete. 2024-06-02T15:49:11.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.968+0000 7efff1b25b80 1 -- start start 2024-06-02T15:49:11.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.968+0000 7efff1b25b80 1 Processor -- start 2024-06-02T15:49:11.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f86717ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86715260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557f8655ffa0 con 0x557f867117c0 2024-06-02T15:49:11.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f865fa760 con 0x557f867120e0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f86721410 con 0x557f8670c390 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.969+0000 7efff1b25b80 10 client.0 _ll_register_callbacks cb 0x557f86603c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34816/0 (socket says 172.21.15.90:34816) 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 -- 172.21.15.90:0/3085548163 learned_addr learned my addr 172.21.15.90:0/3085548163 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effdf400700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86715260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effdfe00700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f86717ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 -- 172.21.15.90:0/3085548163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 msgr2=0x557f86717ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f86717ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 -- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 msgr2=0x557f86715260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86715260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 -- 172.21.15.90:0/3085548163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f865a98d0 con 0x557f867120e0 2024-06-02T15:49:11.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effdf400700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86715260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:11.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.970+0000 7effe5200700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effd80057e0 tx=0x7effd8005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b84430b028a32eb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:11.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7effca000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:11.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8005460 con 0x557f867120e0 2024-06-02T15:49:11.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7effd8023440 con 0x557f867120e0 2024-06-02T15:49:11.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd8004d50 con 0x557f867120e0 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7effd8004b90 con 0x557f867120e0 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7efff1b25b80 10 client.25820 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.971+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f8662a060 con 0x557f867120e0 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd8004d50 con 0x557f867120e0 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7effd8015040 con 0x557f867120e0 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 10 client.25820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:49:11.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 20 client.25820 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f8672dde0 0x557f867301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 4 client.25820 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f8666bd90 con 0x557f8672dde0 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.972+0000 7effdfe00700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f8672dde0 0x557f867301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.973+0000 7effca000700 10 client.25820 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.973+0000 7effdfe00700 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f8672dde0 0x557f867301d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19ce4e5587b8c156 server_cookie=cb95ca7dffee63bd in_seq=0 out_seq=0 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.974+0000 7effca000700 1 -- 172.21.15.90:0/3085548163 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557f8666bd90 con 0x557f8672dde0 2024-06-02T15:49:11.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.974+0000 7effca000700 10 client.25820 handle_command_reply: tid=0 2024-06-02T15:49:11.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.974+0000 7efff1b25b80 1 client.25820 shutdown 2024-06-02T15:49:11.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 msgr2=0x557f867157a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:11.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effd80057e0 tx=0x7effd8005a60 comp rx=0 tx=0).stop 2024-06-02T15:49:11.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 shutdown_connections 2024-06-02T15:49:11.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f8672dde0 0x557f867301d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7effca000700 0 client.25820 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867157a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f8670c390 0x557f86717ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 --2- 172.21.15.90:0/3085548163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f867117c0 0x557f86715260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 >> 172.21.15.90:0/3085548163 conn(0x557f86639be0 msgr2=0x557f86705fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 shutdown_connections 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.975+0000 7efff1b25b80 1 -- 172.21.15.90:0/3085548163 wait complete. 2024-06-02T15:49:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.977+0000 7efff1b25b80 20 client.25820 trim_cache size 0 max 16384 2024-06-02T15:49:11.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.996+0000 7efff1b25b80 1 Processor -- start 2024-06-02T15:49:11.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 -- start start 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86706740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7effe01afad0 con 0x557f8670c390 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe01ad360 con 0x557f867120e0 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7efff1b25b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effe01ab1b0 con 0x557f867117c0 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effe5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effe5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34820/0 (socket says 172.21.15.90:34820) 2024-06-02T15:49:11.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effe5200700 1 -- 172.21.15.90:0/1112027312 learned_addr learned my addr 172.21.15.90:0/1112027312 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:11.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56808/0 (socket says 172.21.15.90:56808) 2024-06-02T15:49:11.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.997+0000 7effdf400700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86706740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:11.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 -- 172.21.15.90:0/1112027312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 msgr2=0x557f86712bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:11.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:11.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 -- 172.21.15.90:0/1112027312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 msgr2=0x557f8670bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 -- 172.21.15.90:0/1112027312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f86711df0 con 0x557f867117c0 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effe5200700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdfe00700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.998+0000 7effdf400700 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86706740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1c22efda05b992 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.999+0000 7effd4c00700 1 -- 172.21.15.90:0/1112027312 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc005070 con 0x557f867117c0 2024-06-02T15:49:12.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.999+0000 7effd4c00700 1 -- 172.21.15.90:0/1112027312 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc027070 con 0x557f867117c0 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:11.999+0000 7effd4c00700 1 -- 172.21.15.90:0/1112027312 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc003d90 con 0x557f867117c0 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 -- 172.21.15.90:0/1112027312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 msgr2=0x557f86706740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86706740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effcc006010 tx=0x7effcc00b260 comp rx=0 tx=0).stop 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 -- 172.21.15.90:0/1112027312 shutdown_connections 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f86712bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f8670bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1112027312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86706740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 -- 172.21.15.90:0/1112027312 >> 172.21.15.90:0/1112027312 conn(0x557f86639be0 msgr2=0x557f867124c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:12.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 -- 172.21.15.90:0/1112027312 shutdown_connections 2024-06-02T15:49:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.000+0000 7efff1b25b80 1 -- 172.21.15.90:0/1112027312 wait complete. 2024-06-02T15:49:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.001+0000 7efff1b25b80 1 -- start start 2024-06-02T15:49:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.001+0000 7efff1b25b80 1 Processor -- start 2024-06-02T15:49:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.001+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:12.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.001+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86735090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:12.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7efff1b25b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:12.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7efff1b25b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7effe01ab820 con 0x557f8670c390 2024-06-02T15:49:12.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7efff1b25b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7effe01acc90 con 0x557f867120e0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7efff1b25b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effe01abef0 con 0x557f867117c0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7efff1b25b80 10 client.0 _ll_register_callbacks cb 0x557f86603c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7effdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7effdfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56812/0 (socket says 172.21.15.90:56812) 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.002+0000 7effdfe00700 1 -- 172.21.15.90:0/1851616450 learned_addr learned my addr 172.21.15.90:0/1851616450 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effe5200700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdf400700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86735090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 -- 172.21.15.90:0/1851616450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 msgr2=0x557f86735090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86735090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 -- 172.21.15.90:0/1851616450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 msgr2=0x557f867355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 -- 172.21.15.90:0/1851616450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f86735d80 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effe5200700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdf400700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86735090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.003+0000 7effdfe00700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effd00a9960 tx=0x7effd00741f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45047f8740903f6a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7effca000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0026040 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7effd00a9c30 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc027070 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7effd0077430 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7efff1b25b80 10 client.15675 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.004+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f86798720 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc027070 con 0x557f8670c390 2024-06-02T15:49:12.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7effd00b23a0 con 0x557f8670c390 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 10 client.15675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 20 client.15675 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f86798f50 0x557f8679b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 4 client.15675 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f86665570 con 0x557f86798f50 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.005+0000 7effdf400700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f86798f50 0x557f8679b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:12.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.006+0000 7effca000700 10 client.15675 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.007+0000 7effdf400700 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f86798f50 0x557f8679b370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d96777f74a4fc389 server_cookie=f38227f8fe17c4e4 in_seq=0 out_seq=0 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7effca000700 1 -- 172.21.15.90:0/1851616450 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f86665570 con 0x557f86798f50 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7effca000700 10 client.15675 handle_command_reply: tid=0 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 client.15675 shutdown 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 msgr2=0x557f86726ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effd00a9960 tx=0x7effd00741f0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 shutdown_connections 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x557f86798f50 0x557f8679b370 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f867120e0 0x557f867355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f8670c390 0x557f86726ce0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effd00a9960 tx=0x7effd00741f0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 --2- 172.21.15.90:0/1851616450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f867117c0 0x557f86735090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7effca000700 0 client.15675 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.008+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 >> 172.21.15.90:0/1851616450 conn(0x557f86639be0 msgr2=0x557f86712d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.009+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 shutdown_connections 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.010+0000 7efff1b25b80 1 -- 172.21.15.90:0/1851616450 wait complete. 2024-06-02T15:49:12.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.010+0000 7efff1b25b80 20 client.15675 trim_cache size 0 max 16384 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 msgr2=0x7effc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 0x7effc40b4dc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effd8020000 tx=0x7effd801d000 comp rx=0 tx=0).stop 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 msgr2=0x7effe019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7effd000e9a0 tx=0x7effd000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 shutdown_connections 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7effc40b2900 0x7effc40b4dc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effe010b650 0x7effe01a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7effe0105650 0x7effe019bd30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 --2- 172.21.15.90:0/4014522622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe0106010 0x7effe019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 >> 172.21.15.90:0/4014522622 conn(0x7effe0101120 msgr2=0x7effe01018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 shutdown_connections 2024-06-02T15:49:12.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:12.012+0000 7effe5c00700 1 -- 172.21.15.90:0/4014522622 wait complete. 2024-06-02T15:49:13.115 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:49:13.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:13 smithi090 ceph-mon[31595]: pgmap v657: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 374 KiB/s wr, 23 op/s 2024-06-02T15:49:13.540 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:49:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:13 smithi120 ceph-mon[36451]: pgmap v657: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 374 KiB/s wr, 23 op/s 2024-06-02T15:49:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:13 smithi135 ceph-mon[36839]: pgmap v657: 129 pgs: 129 active+clean; 5.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 374 KiB/s wr, 23 op/s 2024-06-02T15:49:15.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:15 smithi090 ceph-mon[31595]: pgmap v658: 129 pgs: 129 active+clean; 8.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 604 KiB/s wr, 27 op/s 2024-06-02T15:49:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:15 smithi120 ceph-mon[36451]: pgmap v658: 129 pgs: 129 active+clean; 8.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 604 KiB/s wr, 27 op/s 2024-06-02T15:49:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:15 smithi135 ceph-mon[36839]: pgmap v658: 129 pgs: 129 active+clean; 8.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 604 KiB/s wr, 27 op/s 2024-06-02T15:49:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 -- 172.21.15.90:0/3232405607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c109540 msgr2=0x7fcf7c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 --2- 172.21.15.90:0/3232405607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c109540 0x7fcf7c109920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcf700099c0 tx=0x7fcf70009cd0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 -- 172.21.15.90:0/3232405607 shutdown_connections 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 --2- 172.21.15.90:0/3232405607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c103e50 0x7fcf7c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 --2- 172.21.15.90:0/3232405607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c109540 0x7fcf7c109920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 --2- 172.21.15.90:0/3232405607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103490 0x7fcf7c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 -- 172.21.15.90:0/3232405607 >> 172.21.15.90:0/3232405607 conn(0x7fcf7c0fefc0 msgr2=0x7fcf7c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 -- 172.21.15.90:0/3232405607 shutdown_connections 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.482+0000 7fcf82200700 1 -- 172.21.15.90:0/3232405607 wait complete. 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.483+0000 7fcf82200700 1 Processor -- start 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.483+0000 7fcf82200700 1 -- start start 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 0x7fcf7c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103e50 0x7fcf7c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c113670 con 0x7fcf7c103490 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1134f0 con 0x7fcf7c109540 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf82200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1137f0 con 0x7fcf7c103e50 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 0x7fcf7c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34862/0 (socket says 172.21.15.90:34862) 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 0x7fcf7c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56830/0 (socket says 172.21.15.90:56830) 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 -- 172.21.15.90:0/257366605 learned_addr learned my addr 172.21.15.90:0/257366605 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf7be00700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103e50 0x7fcf7c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 -- 172.21.15.90:0/257366605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103e50 msgr2=0x7fcf7c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103e50 0x7fcf7c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 -- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 msgr2=0x7fcf7c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 0x7fcf7c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.484+0000 7fcf81800700 1 -- 172.21.15.90:0/257366605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1a8030 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf81800700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c370 tx=0x7fcf7400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf79600700 1 -- 172.21.15.90:0/257366605 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740092e0 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1a82c0 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf79600700 1 -- 172.21.15.90:0/257366605 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf74003a00 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf79600700 1 -- 172.21.15.90:0/257366605 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740046f0 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.485+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a87b0 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.488+0000 7fcf79600700 1 -- 172.21.15.90:0/257366605 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcf7401a020 con 0x7fcf7c109540 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.489+0000 7fcf8e168b80 1 Processor -- start 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 -- start start 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf79600700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 0x7fcf640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3db24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3d10e40 con 0x55a5e3db8020 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3d009e0 con 0x55a5e3db2ad0 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3d04a20 con 0x55a5e3db8940 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf79600700 1 -- 172.21.15.90:0/257366605 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcf740e1e40 con 0x7fcf7c109540 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56846/0 (socket says 172.21.15.90:56846) 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf80e00700 1 -- 172.21.15.90:0/561123331 learned_addr learned my addr 172.21.15.90:0/561123331 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf81800700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.490+0000 7fcf7be00700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 0x7fcf640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf7be00700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3db24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 -- 172.21.15.90:0/561123331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 msgr2=0x55a5e3db24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3db24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 -- 172.21.15.90:0/561123331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 msgr2=0x55a5e3db8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 -- 172.21.15.90:0/561123331 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e3cc9420 con 0x55a5e3db2ad0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf80e00700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf81800700 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db2f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf740a8fe0 tx=0x7fcf740a8a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b4e558ada844a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf7be00700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 0x7fcf640b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c108bc0 tx=0x7fcf68007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf6e000700 1 -- 172.21.15.90:0/561123331 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004020 con 0x55a5e3db2ad0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf6e000700 1 -- 172.21.15.90:0/561123331 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740a8b60 con 0x55a5e3db2ad0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.491+0000 7fcf6e000700 1 -- 172.21.15.90:0/561123331 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74019e70 con 0x55a5e3db2ad0 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 -- 172.21.15.90:0/561123331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 msgr2=0x55a5e3db2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db2f30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf740a8fe0 tx=0x7fcf740a8a20 comp rx=0 tx=0).stop 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 -- 172.21.15.90:0/561123331 shutdown_connections 2024-06-02T15:49:16.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db2f30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3db8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/561123331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3db24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 -- 172.21.15.90:0/561123331 >> 172.21.15.90:0/561123331 conn(0x55a5e3ce0be0 msgr2=0x55a5e3dad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 -- 172.21.15.90:0/561123331 shutdown_connections 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.492+0000 7fcf8e168b80 1 -- 172.21.15.90:0/561123331 wait complete. 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 -- start start 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 Processor -- start 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dbe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3dbbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3c06fa0 con 0x55a5e3db8020 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3ca1760 con 0x55a5e3db8940 2024-06-02T15:49:16.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3dbf460 con 0x55a5e3db2ad0 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.493+0000 7fcf8e168b80 10 client.0 _ll_register_callbacks cb 0x55a5e3caac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3dbbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34884/0 (socket says 172.21.15.90:34884) 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 -- 172.21.15.90:0/1801416109 learned_addr learned my addr 172.21.15.90:0/1801416109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dbe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 -- 172.21.15.90:0/1801416109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 msgr2=0x55a5e3dbe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dbe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 -- 172.21.15.90:0/1801416109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 msgr2=0x55a5e3dbbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3dbbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 -- 172.21.15.90:0/1801416109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e3c508d0 con 0x55a5e3db8940 2024-06-02T15:49:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf81800700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf740a8f90 tx=0x7fcf740ad550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85080dc1939318e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf62a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740a92b0 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcf740a9450 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.494+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf740a8b60 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf74004bd0 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 10 client.25838 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5e3cd1060 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740a8b60 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcf740a49c0 con 0x55a5e3db8940 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 20 client.25838 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:16.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3dd46c0 0x55a5e3dd6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 4 client.25838 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e3d12d90 con 0x55a5e3dd46c0 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.495+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3dd46c0 0x55a5e3dd6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.496+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3dd46c0 0x55a5e3dd6ab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fac7847c969fbbb server_cookie=c78eee6fd359238 in_seq=0 out_seq=0 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.496+0000 7fcf62a00700 10 client.25838 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf62a00700 1 -- 172.21.15.90:0/1801416109 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a5e3d12d90 con 0x55a5e3dd46c0 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf62a00700 10 client.25838 handle_command_reply: tid=0 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 client.25838 shutdown 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 msgr2=0x55a5e3dbbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf740a8f90 tx=0x7fcf740ad550 comp rx=0 tx=0).stop 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 shutdown_connections 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3dd46c0 0x55a5e3dd6ab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8940 0x55a5e3dbbfe0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf62a00700 0 client.25838 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1801416109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8020 0x55a5e3dbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 >> 172.21.15.90:0/1801416109 conn(0x55a5e3ce0be0 msgr2=0x55a5e3db14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 shutdown_connections 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.497+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1801416109 wait complete. 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.498+0000 7fcf8e168b80 20 client.25838 trim_cache size 0 max 16384 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.513+0000 7fcf8e168b80 1 Processor -- start 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 -- start start 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8020 0x55a5e3dacc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8940 0x55a5e3dad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1af970 con 0x55a5e3db8940 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad200 con 0x55a5e3db2ad0 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ab050 con 0x55a5e3db8020 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34886/0 (socket says 172.21.15.90:34886) 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf80e00700 1 -- 172.21.15.90:0/1610022429 learned_addr learned my addr 172.21.15.90:0/1610022429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf81800700 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8940 0x55a5e3dad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.514+0000 7fcf7be00700 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8020 0x55a5e3dacc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.515+0000 7fcf80e00700 1 -- 172.21.15.90:0/1610022429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8020 msgr2=0x55a5e3dacc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.515+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8020 0x55a5e3dacc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.515+0000 7fcf80e00700 1 -- 172.21.15.90:0/1610022429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8940 msgr2=0x55a5e3dad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.515+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8940 0x55a5e3dad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.515+0000 7fcf80e00700 1 -- 172.21.15.90:0/1610022429 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e3da7650 con 0x55a5e3db2ad0 2024-06-02T15:49:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.516+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf700099c0 tx=0x7fcf7001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9068b1efd1539f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.516+0000 7fcf6d600700 1 -- 172.21.15.90:0/1610022429 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700037c0 con 0x55a5e3db2ad0 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.516+0000 7fcf6d600700 1 -- 172.21.15.90:0/1610022429 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf70029070 con 0x55a5e3db2ad0 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.516+0000 7fcf6d600700 1 -- 172.21.15.90:0/1610022429 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000be70 con 0x55a5e3db2ad0 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1610022429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 msgr2=0x55a5e3db8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcf700099c0 tx=0x7fcf7001a040 comp rx=0 tx=0).stop 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1610022429 shutdown_connections 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db8940 0x55a5e3dad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db2ad0 0x55a5e3db8400 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1610022429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8020 0x55a5e3dacc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.517+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1610022429 >> 172.21.15.90:0/1610022429 conn(0x55a5e3ce0be0 msgr2=0x55a5e3dbba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.518+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1610022429 shutdown_connections 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.518+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1610022429 wait complete. 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.518+0000 7fcf8e168b80 1 -- start start 2024-06-02T15:49:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.518+0000 7fcf8e168b80 1 Processor -- start 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dd9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3dd7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ab6c0 con 0x55a5e3db2ad0 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1acb30 con 0x55a5e3db8020 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1abd90 con 0x55a5e3db8940 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.519+0000 7fcf8e168b80 10 client.0 _ll_register_callbacks cb 0x55a5e3caac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dd9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf80e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dd9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56872/0 (socket says 172.21.15.90:56872) 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf81800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3dd7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34902/0 (socket says 172.21.15.90:34902) 2024-06-02T15:49:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.520+0000 7fcf7be00700 1 -- 172.21.15.90:0/1395266922 learned_addr learned my addr 172.21.15.90:0/1395266922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 -- 172.21.15.90:0/1395266922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 msgr2=0x55a5e3dd7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3dd7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 -- 172.21.15.90:0/1395266922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 msgr2=0x55a5e3dd9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dd9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 -- 172.21.15.90:0/1395266922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e3dd7ed0 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf7be00700 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007da0 tx=0x7fcf680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74235e5eab856230 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.521+0000 7fcf62a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68004830 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcf680049d0 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf70029070 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf68034050 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf8e168b80 10 client.25844 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.522+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5e3e4fd70 con 0x55a5e3db8020 2024-06-02T15:49:16.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf70029070 con 0x55a5e3db8020 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcf68028070 con 0x55a5e3db8020 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 20 client.25844 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3e50530 0x55a5e3e52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 4 client.25844 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.523+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5e3d0c570 con 0x55a5e3e50530 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.524+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3e50530 0x55a5e3e52950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.524+0000 7fcf80e00700 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3e50530 0x55a5e3e52950 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93fc5d38eaa980d1 server_cookie=73ef2522e3236448 in_seq=0 out_seq=0 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.524+0000 7fcf62a00700 10 client.25844 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.526+0000 7fcf62a00700 1 -- 172.21.15.90:0/1395266922 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5e3d0c570 con 0x55a5e3e50530 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.526+0000 7fcf62a00700 10 client.25844 handle_command_reply: tid=0 2024-06-02T15:49:16.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.526+0000 7fcf8e168b80 1 client.25844 shutdown 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 msgr2=0x55a5e3dd7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007da0 tx=0x7fcf680047c0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 shutdown_connections 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a5e3e50530 0x55a5e3e52950 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e3db8940 0x55a5e3dd7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a5e3db2ad0 0x55a5e3dd9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.595 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 --2- 172.21.15.90:0/1395266922 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a5e3db8020 0x55a5e3dd7350 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 >> 172.21.15.90:0/1395266922 conn(0x55a5e3ce0be0 msgr2=0x55a5e3db5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf62a00700 0 client.25844 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 shutdown_connections 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 1 -- 172.21.15.90:0/1395266922 wait complete. 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.527+0000 7fcf8e168b80 20 client.25844 trim_cache size 0 max 16384 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 msgr2=0x7fcf640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 0x7fcf640b4d40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c108bc0 tx=0x7fcf68007850 comp rx=0 tx=0).stop 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 msgr2=0x7fcf7c0762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:16.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400c370 tx=0x7fcf7400c680 comp rx=0 tx=0).stop 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 shutdown_connections 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcf640b2880 0x7fcf640b4d40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf7c109540 0x7fcf7c0762c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7c103490 0x7fcf7c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 --2- 172.21.15.90:0/257366605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf7c103e50 0x7fcf7c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.529+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 >> 172.21.15.90:0/257366605 conn(0x7fcf7c0fefc0 msgr2=0x7fcf7c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.530+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 shutdown_connections 2024-06-02T15:49:16.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:16.530+0000 7fcf82200700 1 -- 172.21.15.90:0/257366605 wait complete. 2024-06-02T15:49:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 -- 172.21.15.90:0/406102050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 msgr2=0x7fe3a0103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 --2- 172.21.15.90:0/406102050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a0103f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009900 tx=0x7fe38c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 -- 172.21.15.90:0/406102050 shutdown_connections 2024-06-02T15:49:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 --2- 172.21.15.90:0/406102050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 0x7fe3a010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 --2- 172.21.15.90:0/406102050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a0103f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 --2- 172.21.15.90:0/406102050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 -- 172.21.15.90:0/406102050 >> 172.21.15.90:0/406102050 conn(0x7fe3a00fde50 msgr2=0x7fe3a0100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 -- 172.21.15.90:0/406102050 shutdown_connections 2024-06-02T15:49:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.185+0000 7fe39fe00640 1 -- 172.21.15.90:0/406102050 wait complete. 2024-06-02T15:49:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.186+0000 7fe39fe00640 1 Processor -- start 2024-06-02T15:49:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.186+0000 7fe39fe00640 1 -- start start 2024-06-02T15:49:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 0x7fe3a01a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010e440 con 0x7fe3a0104490 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010e2c0 con 0x7fe3a01028e0 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010e5c0 con 0x7fe3a0103ad0 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34910/0 (socket says 172.21.15.90:34910) 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39ea00640 1 -- 172.21.15.90:0/1242766336 learned_addr learned my addr 172.21.15.90:0/1242766336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 0x7fe3a01a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.187+0000 7fe39e000640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 -- 172.21.15.90:0/1242766336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 msgr2=0x7fe3a019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 -- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 msgr2=0x7fe3a01a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 0x7fe3a01a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 -- 172.21.15.90:0/1242766336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a01a2900 con 0x7fe3a01028e0 2024-06-02T15:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe39000e9c0 tx=0x7fe39000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.188+0000 7fe397400640 1 -- 172.21.15.90:0/1242766336 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000cde0 con 0x7fe3a01028e0 2024-06-02T15:49:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.189+0000 7fe397400640 1 -- 172.21.15.90:0/1242766336 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3900108e0 con 0x7fe3a01028e0 2024-06-02T15:49:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.189+0000 7fe397400640 1 -- 172.21.15.90:0/1242766336 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390018980 con 0x7fe3a01028e0 2024-06-02T15:49:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.189+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a01a2bf0 con 0x7fe3a01028e0 2024-06-02T15:49:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.189+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a01a30c0 con 0x7fe3a01028e0 2024-06-02T15:49:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.192+0000 7fe397400640 1 -- 172.21.15.90:0/1242766336 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe39001f020 con 0x7fe3a01028e0 2024-06-02T15:49:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.192+0000 7fe3a8bd0740 1 Processor -- start 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.192+0000 7fe3a8bd0740 1 -- start start 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 0x562ce4fbb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 0x562ce4fc2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ce4f2ab90 con 0x562ce4fbb6e0 2024-06-02T15:49:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562ce4f1a730 con 0x562ce4fbacf0 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ce4f1e770 con 0x562ce4fc0850 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 0x562ce4fbb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 0x562ce4fbb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34920/0 (socket says 172.21.15.90:34920) 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe39f400640 1 -- 172.21.15.90:0/746875476 learned_addr learned my addr 172.21.15.90:0/746875476 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.193+0000 7fe39e000640 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 0x562ce4fc2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39ea00640 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 -- 172.21.15.90:0/746875476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 msgr2=0x562ce4fbb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 0x562ce4fbb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 -- 172.21.15.90:0/746875476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 msgr2=0x562ce4fc01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 -- 172.21.15.90:0/746875476 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce4eb38b0 con 0x562ce4fc0850 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe397400640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 0x7fe3840b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.194+0000 7fe39e000640 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 0x562ce4fc2ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a9e0 tx=0x7fe38c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d793f3d707fe28 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.195+0000 7fe397400640 1 -- 172.21.15.90:0/1242766336 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe3900e27a0 con 0x7fe3a01028e0 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.195+0000 7fe38be00640 1 -- 172.21.15.90:0/746875476 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c035030 con 0x562ce4fc0850 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.195+0000 7fe38be00640 1 -- 172.21.15.90:0/746875476 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38c040070 con 0x562ce4fc0850 2024-06-02T15:49:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.195+0000 7fe38be00640 1 -- 172.21.15.90:0/746875476 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c0318b0 con 0x562ce4fc0850 2024-06-02T15:49:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe39f400640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 0x7fe3840b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/746875476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 msgr2=0x562ce4fc2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 0x562ce4fc2ce0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a9e0 tx=0x7fe38c005c00 comp rx=0 tx=0).stop 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/746875476 shutdown_connections 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fc0850 0x562ce4fc2ce0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/746875476 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fbacf0 0x562ce4fbb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/746875476 >> 172.21.15.90:0/746875476 conn(0x562ce4fb6360 msgr2=0x562ce4fb6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.196+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/746875476 shutdown_connections 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.197+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/746875476 wait complete. 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.197+0000 7fe39f400640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 0x7fe3840b4ca0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x562ce4fbbf40 tx=0x7fe398009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.197+0000 7fe3a8bd0740 1 -- start start 2024-06-02T15:49:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.197+0000 7fe3a8bd0740 1 Processor -- start 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fcbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ce4c00790 con 0x562ce4fbacf0 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562ce4ec2bc0 con 0x562ce4fc0850 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ce4fc37a0 con 0x562ce4fbb6e0 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe3a8bd0740 10 client.0 _ll_register_callbacks cb 0x562ce4ea0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56910/0 (socket says 172.21.15.90:56910) 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.198+0000 7fe39ea00640 1 -- 172.21.15.90:0/947291938 learned_addr learned my addr 172.21.15.90:0/947291938 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39f400640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fcbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39e000640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 -- 172.21.15.90:0/947291938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 msgr2=0x562ce4fc4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 -- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 msgr2=0x562ce4fcbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fcbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 -- 172.21.15.90:0/947291938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce4c2f210 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39e000640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe39ea00640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3900a5a30 tx=0x7fe3900a93f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc974ef0912cb53 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe388c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900aa0b0 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe390010a90 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.199+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38c040070 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3900a99b0 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe3a8bd0740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ce4cfb620 con 0x562ce4fbacf0 2024-06-02T15:49:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38c040070 con 0x562ce4fbacf0 2024-06-02T15:49:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe3900c6050 con 0x562ce4fbacf0 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe3a8bd0740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.200+0000 7fe3a8bd0740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe3a8bd0740 20 client.15705 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce4fd0590 0x562ce4fd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe3a8bd0740 4 client.15705 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce4f2cae0 con 0x562ce4fd0590 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe39e000640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce4fd0590 0x562ce4fd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe39e000640 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce4fd0590 0x562ce4fd2a00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471b21436fde778c server_cookie=2813e71a4503c7ec in_seq=0 out_seq=0 2024-06-02T15:49:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.201+0000 7fe388c00640 10 client.15705 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe388c00640 1 -- 172.21.15.90:0/947291938 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ce4f2cae0 con 0x562ce4fd0590 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe388c00640 10 client.15705 handle_command_reply: tid=0 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 client.15705 shutdown 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 msgr2=0x562ce4fc4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3900a5a30 tx=0x7fe3900a93f0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 shutdown_connections 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce4fd0590 0x562ce4fd2a00 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fcbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/947291938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fc4a50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 >> 172.21.15.90:0/947291938 conn(0x562ce4fb6360 msgr2=0x562ce4fb54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe388c00640 0 client.15705 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 shutdown_connections 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/947291938 wait complete. 2024-06-02T15:49:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.205+0000 7fe3a8bd0740 20 client.15705 trim_cache size 0 max 16384 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.222+0000 7fe3a8bd0740 1 Processor -- start 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 -- start start 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fcbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fc4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01aff10 con 0x562ce4fbacf0 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01ad7e0 con 0x562ce4fc0850 2024-06-02T15:49:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01ab630 con 0x562ce4fbb6e0 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fc4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56916/0 (socket says 172.21.15.90:56916) 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fc4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34928/0 (socket says 172.21.15.90:34928) 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39ea00640 1 -- 172.21.15.90:0/1103885621 learned_addr learned my addr 172.21.15.90:0/1103885621 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.223+0000 7fe39e000640 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fcbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 -- 172.21.15.90:0/1103885621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 msgr2=0x562ce4fcbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fcbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 -- 172.21.15.90:0/1103885621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 msgr2=0x562ce4fc4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fc4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 -- 172.21.15.90:0/1103885621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce4fb6190 con 0x562ce4fbacf0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3900a9660 tx=0x7fe3900a58b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf6c400a5ac0e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe38b400640 1 -- 172.21.15.90:0/1103885621 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900aa0b0 con 0x562ce4fbacf0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe38b400640 1 -- 172.21.15.90:0/1103885621 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3900b2380 con 0x562ce4fbacf0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.224+0000 7fe38b400640 1 -- 172.21.15.90:0/1103885621 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3900e2450 con 0x562ce4fbacf0 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.225+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1103885621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 msgr2=0x562ce4fb9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.225+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3900a9660 tx=0x7fe3900a58b0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.226+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1103885621 shutdown_connections 2024-06-02T15:49:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.226+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce4fc4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.226+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbb6e0 0x562ce4fcbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.226+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1103885621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbacf0 0x562ce4fb9980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.226+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1103885621 >> 172.21.15.90:0/1103885621 conn(0x562ce4fb6360 msgr2=0x562ce4fb9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1103885621 shutdown_connections 2024-06-02T15:49:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1103885621 wait complete. 2024-06-02T15:49:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- start start 2024-06-02T15:49:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 Processor -- start 2024-06-02T15:49:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbacf0 0x562ce4fc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01abca0 con 0x562ce4fbb6e0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01ad110 con 0x562ce4fc0850 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01ac370 con 0x562ce4fbacf0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 10 client.0 _ll_register_callbacks cb 0x7fe3900a97f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34938/0 (socket says 172.21.15.90:34938) 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 -- 172.21.15.90:0/1984879874 learned_addr learned my addr 172.21.15.90:0/1984879874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39e000640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbacf0 0x562ce4fc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 -- 172.21.15.90:0/1984879874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbacf0 msgr2=0x562ce4fc8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbacf0 0x562ce4fc8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 -- 172.21.15.90:0/1984879874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 msgr2=0x562ce4fc93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 -- 172.21.15.90:0/1984879874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce5039710 con 0x562ce4fc0850 2024-06-02T15:49:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39f400640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe39800fcb0 tx=0x7fe398004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e686201fc5392633 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39801dba0 con 0x562ce4fc0850 2024-06-02T15:49:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3900b2380 con 0x562ce4fc0850 2024-06-02T15:49:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe39801dd40 con 0x562ce4fc0850 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe39804a050 con 0x562ce4fc0850 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 10 client.25862 fetch_fsmap learned FSMap version 30 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562ce503a8b0 con 0x562ce4fc0850 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3900b2380 con 0x562ce4fc0850 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe398043470 con 0x562ce4fc0850 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 10 client.25862 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 20 client.25862 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce503b310 0x562ce503d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 4 client.25862 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ce4f262c0 con 0x562ce503b310 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.228+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce503b310 0x562ce503d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.234+0000 7fe388c00640 10 client.25862 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:17.234+0000 7fe39ea00640 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce503b310 0x562ce503d730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729979cea9b39efc server_cookie=8aba79c60639df14 in_seq=0 out_seq=0 2024-06-02T15:49:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:17 smithi120 ceph-mon[36451]: pgmap v659: 129 pgs: 129 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-06-02T15:49:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:17 smithi135 ceph-mon[36839]: pgmap v659: 129 pgs: 129 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-06-02T15:49:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:17 smithi090 ceph-mon[31595]: pgmap v659: 129 pgs: 129 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-06-02T15:49:18.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:49:18.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:49:19.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:19 smithi090 ceph-mon[31595]: pgmap v660: 129 pgs: 129 active+clean; 19 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 72 op/s 2024-06-02T15:49:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:19 smithi120 ceph-mon[36451]: pgmap v660: 129 pgs: 129 active+clean; 19 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 72 op/s 2024-06-02T15:49:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:19 smithi135 ceph-mon[36839]: pgmap v660: 129 pgs: 129 active+clean; 19 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 72 op/s 2024-06-02T15:49:19.687 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:49:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:21 smithi120 ceph-mon[36451]: pgmap v661: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 347 op/s 2024-06-02T15:49:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:21 smithi135 ceph-mon[36839]: pgmap v661: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 347 op/s 2024-06-02T15:49:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:21 smithi090 ceph-mon[31595]: pgmap v661: 129 pgs: 129 active+clean; 19 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 347 op/s 2024-06-02T15:49:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe388c00640 1 -- 172.21.15.90:0/1984879874 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562ce4f262c0 con 0x562ce503b310 2024-06-02T15:49:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe388c00640 10 client.25862 handle_command_reply: tid=0 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 client.25862 shutdown 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 msgr2=0x562ce50391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe39800fcb0 tx=0x7fe398004ab0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 shutdown_connections 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562ce503b310 0x562ce503d730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ce4fc0850 0x562ce50391d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ce4fbb6e0 0x562ce4fc93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 --2- 172.21.15.90:0/1984879874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ce4fbacf0 0x562ce4fc8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe388c00640 0 client.25862 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 >> 172.21.15.90:0/1984879874 conn(0x562ce4fb6360 msgr2=0x562ce4fbf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 shutdown_connections 2024-06-02T15:49:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.930+0000 7fe3a8bd0740 1 -- 172.21.15.90:0/1984879874 wait complete. 2024-06-02T15:49:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe3a8bd0740 20 client.25862 trim_cache size 0 max 16384 2024-06-02T15:49:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:49:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:49:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 msgr2=0x7fe3840b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 0x7fe3840b4ca0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x562ce4fbbf40 tx=0x7fe398009500 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 msgr2=0x7fe3a019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe39000e9c0 tx=0x7fe39000ee80 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 shutdown_connections 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3840b27e0 0x7fe3840b4ca0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3a0104490 0x7fe3a01a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0103ad0 0x7fe3a019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 --2- 172.21.15.90:0/1242766336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe3a01028e0 0x7fe3a019ad60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.931+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 >> 172.21.15.90:0/1242766336 conn(0x7fe3a00fde50 msgr2=0x7fe3a0074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.932+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 shutdown_connections 2024-06-02T15:49:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:21.932+0000 7fe39fe00640 1 -- 172.21.15.90:0/1242766336 wait complete. 2024-06-02T15:49:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:23 smithi120 ceph-mon[36451]: pgmap v662: 129 pgs: 129 active+clean; 19 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 1.14k op/s 2024-06-02T15:49:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:23 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:23 smithi135 ceph-mon[36839]: pgmap v662: 129 pgs: 129 active+clean; 19 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 1.14k op/s 2024-06-02T15:49:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:23 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:23 smithi090 ceph-mon[31595]: pgmap v662: 129 pgs: 129 active+clean; 19 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 1.14k op/s 2024-06-02T15:49:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:23 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:24.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 -- 172.21.15.90:0/2706083386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4077a70 msgr2=0x7feca4077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 --2- 172.21.15.90:0/2706083386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4077a70 0x7feca4077e50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec98009990 tx=0x7fec98009ca0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 -- 172.21.15.90:0/2706083386 shutdown_connections 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 --2- 172.21.15.90:0/2706083386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41143e0 0x7feca41167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 --2- 172.21.15.90:0/2706083386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4077a70 0x7feca4077e50 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec98009990 tx=0x7fec98009ca0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 --2- 172.21.15.90:0/2706083386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca4078390 0x7feca4113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.234+0000 7fecab600700 1 -- 172.21.15.90:0/2706083386 >> 172.21.15.90:0/2706083386 conn(0x7feca4101250 msgr2=0x7feca4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.236+0000 7fecab600700 1 -- 172.21.15.90:0/2706083386 shutdown_connections 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.236+0000 7fecab600700 1 -- 172.21.15.90:0/2706083386 wait complete. 2024-06-02T15:49:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.237+0000 7fecab600700 1 Processor -- start 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.237+0000 7fecab600700 1 -- start start 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca41143e0 0x7feca41a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41a4d60 0x7feca41a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feca4119db0 con 0x7feca41143e0 2024-06-02T15:49:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feca4119c30 con 0x7feca4078390 2024-06-02T15:49:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7fecab600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca4119f30 con 0x7feca41a4d60 2024-06-02T15:49:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39114/0 (socket says 172.21.15.90:39114) 2024-06-02T15:49:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7feca8c00700 1 -- 172.21.15.90:0/545344504 learned_addr learned my addr 172.21.15.90:0/545344504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.238+0000 7feca9600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41a4d60 0x7feca41a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca3e00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca41143e0 0x7feca41a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 -- 172.21.15.90:0/545344504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41a4d60 msgr2=0x7feca41a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41a4d60 0x7feca41a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 -- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca41143e0 msgr2=0x7feca41a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca41143e0 0x7feca41a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 -- 172.21.15.90:0/545344504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41a5880 con 0x7feca4078390 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.239+0000 7feca8c00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fec98015000 tx=0x7fec980057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.240+0000 7feca1600700 1 -- 172.21.15.90:0/545344504 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98026070 con 0x7feca4078390 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.240+0000 7feca1600700 1 -- 172.21.15.90:0/545344504 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec98021410 con 0x7feca4078390 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.240+0000 7feca1600700 1 -- 172.21.15.90:0/545344504 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98017650 con 0x7feca4078390 2024-06-02T15:49:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.240+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41a5b10 con 0x7feca4078390 2024-06-02T15:49:24.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.240+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41ad1e0 con 0x7feca4078390 2024-06-02T15:49:24.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.245+0000 7feca1600700 1 -- 172.21.15.90:0/545344504 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec98009d10 con 0x7feca4078390 2024-06-02T15:49:24.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.247+0000 7feca1600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 0x7fec8c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.248+0000 7feca1600700 1 -- 172.21.15.90:0/545344504 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fec980e29b0 con 0x7feca4078390 2024-06-02T15:49:24.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.249+0000 7feca3e00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 0x7fec8c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.249+0000 7fecb5ed8b80 1 Processor -- start 2024-06-02T15:49:24.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 -- start start 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f152d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f4f0b0ec0 con 0x561f4f152900 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f4f0a0a60 con 0x561f4f157f30 2024-06-02T15:49:24.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f4f0a4aa0 con 0x561f4f158850 2024-06-02T15:49:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39128/0 (socket says 172.21.15.90:39128) 2024-06-02T15:49:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7feca8c00700 1 -- 172.21.15.90:0/2683835138 learned_addr learned my addr 172.21.15.90:0/2683835138 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:24.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.250+0000 7feca9600700 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca8c00700 1 -- 172.21.15.90:0/2683835138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 msgr2=0x561f4f1522e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca8c00700 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f1522e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca8c00700 1 -- 172.21.15.90:0/2683835138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 msgr2=0x561f4f152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca8c00700 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca8c00700 1 -- 172.21.15.90:0/2683835138 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4f0694a0 con 0x561f4f157f30 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.251+0000 7feca9600700 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.252+0000 7feca8c00700 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec98009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83ed1ed2798a575 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.252+0000 7fec92000700 1 -- 172.21.15.90:0/2683835138 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980b2370 con 0x561f4f157f30 2024-06-02T15:49:24.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.252+0000 7fec92000700 1 -- 172.21.15.90:0/2683835138 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec980aa350 con 0x561f4f157f30 2024-06-02T15:49:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.252+0000 7fec92000700 1 -- 172.21.15.90:0/2683835138 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980218e0 con 0x561f4f157f30 2024-06-02T15:49:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.253+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/2683835138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 msgr2=0x561f4f158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.253+0000 7feca3e00700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 0x7fec8c0b4e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feca407a730 tx=0x7fec94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.253+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec98009310 comp rx=0 tx=0).stop 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/2683835138 shutdown_connections 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f158310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/2683835138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f1522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/2683835138 >> 172.21.15.90:0/2683835138 conn(0x561f4f080c60 msgr2=0x561f4f14d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/2683835138 shutdown_connections 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/2683835138 wait complete. 2024-06-02T15:49:24.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.254+0000 7fecb5ed8b80 1 -- start start 2024-06-02T15:49:24.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 Processor -- start 2024-06-02T15:49:24.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f15e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f158850 0x561f4f15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f4efa7020 con 0x561f4f158850 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.255+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561f4f0417e0 con 0x561f4f152900 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f4f15f080 con 0x561f4f157f30 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7fecb5ed8b80 10 client.0 _ll_register_callbacks cb 0x561f4f04acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7feca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7feca3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38488/0 (socket says 172.21.15.90:38488) 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7feca3e00700 1 -- 172.21.15.90:0/1407956470 learned_addr learned my addr 172.21.15.90:0/1407956470 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7feca8c00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.256+0000 7feca9600700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f158850 0x561f4f15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 -- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 msgr2=0x561f4f15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 -- 172.21.15.90:0/1407956470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f158850 msgr2=0x561f4f15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f158850 0x561f4f15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 -- 172.21.15.90:0/1407956470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f4eff0950 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7feca3e00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec94005910 tx=0x7fec940047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e8f7a10a11b1fd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7fec86a00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94004220 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fec940043c0 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.257+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec980aa350 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.258+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fec9402c050 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.258+0000 7fecb5ed8b80 10 client.25828 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.258+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f4f0710e0 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec980aa350 con 0x561f4f157f30 2024-06-02T15:49:24.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fec9401f9b0 con 0x561f4f157f30 2024-06-02T15:49:24.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:24.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:24.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 20 client.25828 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f17cd20 0x561f4f17f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 4 client.25828 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.259+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f4f0b2e10 con 0x561f4f17cd20 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.260+0000 7feca8c00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f17cd20 0x561f4f17f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.260+0000 7fec86a00700 10 client.25828 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:24.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.261+0000 7feca8c00700 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f17cd20 0x561f4f17f180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0dcfd5b95a093e server_cookie=69c20b11ddc31dee in_seq=0 out_seq=0 2024-06-02T15:49:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.261+0000 7fec86a00700 1 -- 172.21.15.90:0/1407956470 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f4f0b2e10 con 0x561f4f17cd20 2024-06-02T15:49:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.261+0000 7fec86a00700 10 client.25828 handle_command_reply: tid=0 2024-06-02T15:49:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.262+0000 7fecb5ed8b80 1 client.25828 shutdown 2024-06-02T15:49:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 msgr2=0x561f4f15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec94005910 tx=0x7fec940047c0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 shutdown_connections 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f17cd20 0x561f4f17f180 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f158850 0x561f4f15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/1407956470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f157f30 0x561f4f15ba00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 >> 172.21.15.90:0/1407956470 conn(0x561f4f080c60 msgr2=0x561f4f1512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fec86a00700 0 client.25828 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:24.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 shutdown_connections 2024-06-02T15:49:24.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/1407956470 wait complete. 2024-06-02T15:49:24.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.263+0000 7fecb5ed8b80 20 client.25828 trim_cache size 0 max 16384 2024-06-02T15:49:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.291+0000 7fecb5ed8b80 1 Processor -- start 2024-06-02T15:49:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.291+0000 7fecb5ed8b80 1 -- start start 2024-06-02T15:49:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f184a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feca41aff10 con 0x561f4f152900 2024-06-02T15:49:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feca41ad7a0 con 0x561f4f157f30 2024-06-02T15:49:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca4064950 con 0x561f4f158850 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7feca3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7feca3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39142/0 (socket says 172.21.15.90:39142) 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.292+0000 7feca3e00700 1 -- 172.21.15.90:0/3473124667 learned_addr learned my addr 172.21.15.90:0/3473124667 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca9600700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f184a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 -- 172.21.15.90:0/3473124667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 msgr2=0x561f4f184a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f184a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 -- 172.21.15.90:0/3473124667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 msgr2=0x561f4f1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca8c00700 1 -- 172.21.15.90:0/3473124667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4f14ddc0 con 0x561f4f152900 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca9600700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f184a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.293+0000 7feca3e00700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.294+0000 7feca8c00700 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f1522e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fec980ae1c0 tx=0x7fec98005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b411041cc208cd47 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.294+0000 7fec91600700 1 -- 172.21.15.90:0/3473124667 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980be070 con 0x561f4f152900 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.295+0000 7fec91600700 1 -- 172.21.15.90:0/3473124667 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec980a9c50 con 0x561f4f152900 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.295+0000 7fec91600700 1 -- 172.21.15.90:0/3473124667 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980b6070 con 0x561f4f152900 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3473124667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 msgr2=0x561f4f1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f1522e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fec980ae1c0 tx=0x7fec98005040 comp rx=0 tx=0).stop 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3473124667 shutdown_connections 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f184a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f152900 0x561f4f1522e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3473124667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f157f30 0x561f4f1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3473124667 >> 172.21.15.90:0/3473124667 conn(0x561f4f080c60 msgr2=0x561f4f15be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3473124667 shutdown_connections 2024-06-02T15:49:24.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.296+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3473124667 wait complete. 2024-06-02T15:49:24.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.297+0000 7fecb5ed8b80 1 -- start start 2024-06-02T15:49:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.297+0000 7fecb5ed8b80 1 Processor -- start 2024-06-02T15:49:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.297+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.297+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 0x561f4f1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7fecb5ed8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f161d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feca41a17f0 con 0x561f4f157f30 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feca410f3d0 con 0x561f4f152900 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7fecb5ed8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca410e630 con 0x561f4f158850 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7fecb5ed8b80 10 client.0 _ll_register_callbacks cb 0x561f4f04acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:39144/0 (socket says 172.21.15.90:39144) 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 -- 172.21.15.90:0/3742667223 learned_addr learned my addr 172.21.15.90:0/3742667223 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca9600700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca3e00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 0x561f4f1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 -- 172.21.15.90:0/3742667223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 msgr2=0x561f4f161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 -- 172.21.15.90:0/3742667223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 msgr2=0x561f4f1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 0x561f4f1617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca8c00700 1 -- 172.21.15.90:0/3742667223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f4f1e7930 con 0x561f4f152900 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.298+0000 7feca3e00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 0x561f4f1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.299+0000 7feca8c00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec98005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9e504e88dc656 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.299+0000 7fec86a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.299+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec980b6030 con 0x561f4f152900 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.299+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fec980a6240 con 0x561f4f152900 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.299+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f4f0ac5f0 con 0x561f4f152900 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.300+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fec980be040 con 0x561f4f152900 2024-06-02T15:49:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.300+0000 7fecb5ed8b80 10 client.25880 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.300+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f4f1e8bd0 con 0x561f4f152900 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.301+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f4f0ac5f0 con 0x561f4f152900 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fec980b2050 con 0x561f4f152900 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 10 client.25880 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 20 client.25880 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f1e8e00 0x561f4f1eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 4 client.25880 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:49:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f4ef232d0 con 0x561f4f1e8e00 2024-06-02T15:49:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.302+0000 7feca3e00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f1e8e00 0x561f4f1eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.303+0000 7fec86a00700 10 client.25880 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.303+0000 7feca3e00700 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f1e8e00 0x561f4f1eb1f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8861700d48ba84a server_cookie=b07df4e6f8bdc200 in_seq=0 out_seq=0 2024-06-02T15:49:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.304+0000 7fec86a00700 1 -- 172.21.15.90:0/3742667223 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f4ef232d0 con 0x561f4f1e8e00 2024-06-02T15:49:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.304+0000 7fec86a00700 10 client.25880 handle_command_reply: tid=0 2024-06-02T15:49:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.304+0000 7fecb5ed8b80 1 client.25880 shutdown 2024-06-02T15:49:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.304+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 msgr2=0x561f4f1612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.304+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c00 tx=0x7fec98005fb0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 shutdown_connections 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561f4f1e8e00 0x561f4f1eb1f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f4f158850 0x561f4f161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561f4f152900 0x561f4f1612a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 --2- 172.21.15.90:0/3742667223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f4f157f30 0x561f4f1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 >> 172.21.15.90:0/3742667223 conn(0x561f4f080c60 msgr2=0x561f4f15a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.305+0000 7fec86a00700 0 client.25880 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:24.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.306+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 shutdown_connections 2024-06-02T15:49:24.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.306+0000 7fecb5ed8b80 1 -- 172.21.15.90:0/3742667223 wait complete. 2024-06-02T15:49:24.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.306+0000 7fecb5ed8b80 20 client.25880 trim_cache size 0 max 16384 2024-06-02T15:49:24.305 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:49:24.305 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:49:24.305 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "9744a362-141c-4314-b3c3-3115ac9f31b3", 2024-06-02T15:49:24.305 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:49:24.305 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 msgr2=0x7fec8c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 0x7fec8c0b4e90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feca407a730 tx=0x7fec94009260 comp rx=0 tx=0).stop 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 msgr2=0x7feca41a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fec98015000 tx=0x7fec980057c0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 shutdown_connections 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fec8c0b29d0 0x7fec8c0b4e90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca41a4d60 0x7feca41a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feca4078390 0x7feca41a0490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 --2- 172.21.15.90:0/545344504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feca41143e0 0x7feca41a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.308+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 >> 172.21.15.90:0/545344504 conn(0x7feca4101250 msgr2=0x7feca4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.310+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 shutdown_connections 2024-06-02T15:49:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:24.310+0000 7fecab600700 1 -- 172.21.15.90:0/545344504 wait complete. 2024-06-02T15:49:25.268 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: pgmap v663: 129 pgs: 129 active+clean; 20 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 1.54k op/s 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:25 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: pgmap v663: 129 pgs: 129 active+clean; 20 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 1.54k op/s 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:25 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: pgmap v663: 129 pgs: 129 active+clean; 20 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 1.54k op/s 2024-06-02T15:49:25.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:49:25.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:25 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:25.848 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:49:26.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:26 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:26 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:26 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:27 smithi120 ceph-mon[36451]: pgmap v664: 129 pgs: 129 active+clean; 16 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-06-02T15:49:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:27 smithi135 ceph-mon[36839]: pgmap v664: 129 pgs: 129 active+clean; 16 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-06-02T15:49:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:27 smithi090 ceph-mon[31595]: pgmap v664: 129 pgs: 129 active+clean; 16 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-06-02T15:49:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:29 smithi120 ceph-mon[36451]: pgmap v665: 129 pgs: 129 active+clean; 13 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 2.16k op/s 2024-06-02T15:49:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:29 smithi135 ceph-mon[36839]: pgmap v665: 129 pgs: 129 active+clean; 13 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 2.16k op/s 2024-06-02T15:49:29.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:29 smithi090 ceph-mon[31595]: pgmap v665: 129 pgs: 129 active+clean; 13 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 2.16k op/s 2024-06-02T15:49:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:31 smithi120 ceph-mon[36451]: pgmap v666: 129 pgs: 129 active+clean; 18 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 2.27k op/s 2024-06-02T15:49:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:31 smithi135 ceph-mon[36839]: pgmap v666: 129 pgs: 129 active+clean; 18 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 2.27k op/s 2024-06-02T15:49:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:31 smithi090 ceph-mon[31595]: pgmap v666: 129 pgs: 129 active+clean; 18 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 2.27k op/s 2024-06-02T15:49:32.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:32 smithi120 ceph-mon[36451]: pgmap v667: 129 pgs: 129 active+clean; 18 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.04k op/s 2024-06-02T15:49:32.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:32 smithi135 ceph-mon[36839]: pgmap v667: 129 pgs: 129 active+clean; 18 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.04k op/s 2024-06-02T15:49:32.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:32 smithi090 ceph-mon[31595]: pgmap v667: 129 pgs: 129 active+clean; 18 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 2.04k op/s 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 -- 172.21.15.90:0/3847115714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c108500 msgr2=0x7f309c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 --2- 172.21.15.90:0/3847115714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c108500 0x7f309c1088e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f308c009990 tx=0x7f308c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 -- 172.21.15.90:0/3847115714 shutdown_connections 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 --2- 172.21.15.90:0/3847115714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c103f90 0x7f309c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 --2- 172.21.15.90:0/3847115714 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c108500 0x7f309c1088e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 --2- 172.21.15.90:0/3847115714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f309c1035f0 0x7f309c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 -- 172.21.15.90:0/3847115714 >> 172.21.15.90:0/3847115714 conn(0x7f309c07a6a0 msgr2=0x7f309c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 -- 172.21.15.90:0/3847115714 shutdown_connections 2024-06-02T15:49:34.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.508+0000 7f30a2c00700 1 -- 172.21.15.90:0/3847115714 wait complete. 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.510+0000 7f30a2c00700 1 Processor -- start 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.510+0000 7f30a2c00700 1 -- start start 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f309c1035f0 0x7f309c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c108500 0x7f309c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f309c115aa0 con 0x7f309c1035f0 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f309c115920 con 0x7f309c103f90 2024-06-02T15:49:34.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.511+0000 7f30a2c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f309c115c20 con 0x7f309c108500 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.512+0000 7f309b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.512+0000 7f309b400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:32964/0 (socket says 172.21.15.90:32964) 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.512+0000 7f309b400700 1 -- 172.21.15.90:0/712383323 learned_addr learned my addr 172.21.15.90:0/712383323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.512+0000 7f30a0c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c108500 0x7f309c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 -- 172.21.15.90:0/712383323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c108500 msgr2=0x7f309c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c108500 0x7f309c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 -- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f309c1035f0 msgr2=0x7f309c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f309c1035f0 0x7f309c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 -- 172.21.15.90:0/712383323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c1a0ec0 con 0x7f309c103f90 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f309b400700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f308800e9a0 tx=0x7f308800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f3098c00700 1 -- 172.21.15.90:0/712383323 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308800cbd0 con 0x7f309c103f90 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f3098c00700 1 -- 172.21.15.90:0/712383323 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308800cd70 con 0x7f309c103f90 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.513+0000 7f3098c00700 1 -- 172.21.15.90:0/712383323 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088018680 con 0x7f309c103f90 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.514+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c1a11b0 con 0x7f309c103f90 2024-06-02T15:49:34.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.514+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c110cb0 con 0x7f309c103f90 2024-06-02T15:49:34.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.520+0000 7f30ad4beb80 1 Processor -- start 2024-06-02T15:49:34.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.521+0000 7f3098c00700 1 -- 172.21.15.90:0/712383323 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3088010430 con 0x7f309c103f90 2024-06-02T15:49:34.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.522+0000 7f30ad4beb80 1 -- start start 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f352260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 0x55824f352ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55824f2b0e40 con 0x55824f357eb0 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55824f2a09e0 con 0x55824f352880 2024-06-02T15:49:34.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55824f2a4a20 con 0x55824f3587d0 2024-06-02T15:49:34.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f309be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f352260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45550/0 (socket says 172.21.15.90:45550) 2024-06-02T15:49:34.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.523+0000 7f30a0c00700 1 -- 172.21.15.90:0/1521506039 learned_addr learned my addr 172.21.15.90:0/1521506039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:34.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.524+0000 7f3098c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 0x7f30800b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.525+0000 7f3098c00700 1 -- 172.21.15.90:0/712383323 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f30880e2790 con 0x7f309c103f90 2024-06-02T15:49:34.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.525+0000 7f30a0c00700 1 -- 172.21.15.90:0/1521506039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 msgr2=0x55824f352260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f309b400700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 0x55824f352ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f30a0c00700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f352260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f30a0c00700 1 -- 172.21.15.90:0/1521506039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 msgr2=0x55824f352ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f30a0c00700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 0x55824f352ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f30a0c00700 1 -- 172.21.15.90:0/1521506039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824f269420 con 0x55824f357eb0 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f30a0c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 0x7f30800b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f309be00700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f352260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.526+0000 7f309b400700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 0x55824f352ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.527+0000 7f30a0c00700 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3094015960 tx=0x7f3094015c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54b31ad422a168 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.527+0000 7f30a0c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 0x7f30800b4e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3094022000 tx=0x7f3094024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.528+0000 7f3091600700 1 -- 172.21.15.90:0/1521506039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094007700 con 0x55824f357eb0 2024-06-02T15:49:34.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.528+0000 7f3091600700 1 -- 172.21.15.90:0/1521506039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3094007d50 con 0x55824f357eb0 2024-06-02T15:49:34.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.528+0000 7f3091600700 1 -- 172.21.15.90:0/1521506039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094021d50 con 0x55824f357eb0 2024-06-02T15:49:34.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 -- 172.21.15.90:0/1521506039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 msgr2=0x55824f358290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3094015960 tx=0x7f3094015c70 comp rx=0 tx=0).stop 2024-06-02T15:49:34.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 -- 172.21.15.90:0/1521506039 shutdown_connections 2024-06-02T15:49:34.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f352880 0x55824f352ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f358290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/1521506039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f352260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.529+0000 7f30ad4beb80 1 -- 172.21.15.90:0/1521506039 >> 172.21.15.90:0/1521506039 conn(0x55824f280be0 msgr2=0x55824f34d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.530+0000 7f30ad4beb80 1 -- 172.21.15.90:0/1521506039 shutdown_connections 2024-06-02T15:49:34.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.530+0000 7f30ad4beb80 1 -- 172.21.15.90:0/1521506039 wait complete. 2024-06-02T15:49:34.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.533+0000 7f30ad4beb80 1 -- start start 2024-06-02T15:49:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.534+0000 7f30ad4beb80 1 Processor -- start 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f35e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 0x55824f35b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55824f1a6fa0 con 0x55824f352880 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55824f241760 con 0x55824f357eb0 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55824f35f240 con 0x55824f3587d0 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.535+0000 7f30ad4beb80 10 client.0 _ll_register_callbacks cb 0x55824f24ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.536+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.536+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:43714/0 (socket says 172.21.15.90:43714) 2024-06-02T15:49:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.536+0000 7f30a0c00700 1 -- 172.21.15.90:0/567430831 learned_addr learned my addr 172.21.15.90:0/567430831 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f309b400700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 0x55824f35b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 -- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 msgr2=0x55824f35b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 0x55824f35b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 -- 172.21.15.90:0/567430831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 msgr2=0x55824f35e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f35e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 -- 172.21.15.90:0/567430831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824f1f08d0 con 0x55824f3587d0 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30a0c00700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3094009550 tx=0x7f3094021650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b95ffe5bb4f086f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f3086000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f309b400700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 0x55824f35b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940155f0 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094007d50 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.537+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3094007710 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.538+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3094039050 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.538+0000 7f30ad4beb80 10 client.25843 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.538+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55824f271060 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.538+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3094007d50 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.538+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f309402a960 con 0x55824f3587d0 2024-06-02T15:49:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 10 client.25843 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 20 client.25843 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f37cbe0 0x55824f37efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 4 client.25843 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55824f2b2d90 con 0x55824f37cbe0 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.539+0000 7f309be00700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f37cbe0 0x55824f37efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.540+0000 7f309be00700 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f37cbe0 0x55824f37efd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03bf79021070196 server_cookie=550cd3b026099eb6 in_seq=0 out_seq=0 2024-06-02T15:49:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.540+0000 7f3086000700 10 client.25843 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f3086000700 1 -- 172.21.15.90:0/567430831 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55824f2b2d90 con 0x55824f37cbe0 2024-06-02T15:49:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f3086000700 10 client.25843 handle_command_reply: tid=0 2024-06-02T15:49:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 client.25843 shutdown 2024-06-02T15:49:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 msgr2=0x55824f35bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3094009550 tx=0x7f3094021650 comp rx=0 tx=0).stop 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 shutdown_connections 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f37cbe0 0x55824f37efd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f3587d0 0x55824f35bdd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f35e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/567430831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f357eb0 0x55824f35b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.541+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 >> 172.21.15.90:0/567430831 conn(0x55824f280be0 msgr2=0x55824f351250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.542+0000 7f3086000700 0 client.25843 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.542+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 shutdown_connections 2024-06-02T15:49:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.543+0000 7f30ad4beb80 1 -- 172.21.15.90:0/567430831 wait complete. 2024-06-02T15:49:34.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.543+0000 7f30ad4beb80 20 client.25843 trim_cache size 0 max 16384 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.577+0000 7f30ad4beb80 1 Processor -- start 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.577+0000 7f30ad4beb80 1 -- start start 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f352880 0x55824f352260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f35bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1afba0 con 0x55824f357eb0 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1ad430 con 0x55824f3587d0 2024-06-02T15:49:34.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1ab280 con 0x55824f352880 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33002/0 (socket says 172.21.15.90:33002) 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.578+0000 7f30a0c00700 1 -- 172.21.15.90:0/2368940407 learned_addr learned my addr 172.21.15.90:0/2368940407 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.579+0000 7f30a0c00700 1 -- 172.21.15.90:0/2368940407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f352880 msgr2=0x55824f352260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.579+0000 7f30a0c00700 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f352880 0x55824f352260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.579+0000 7f30a0c00700 1 -- 172.21.15.90:0/2368940407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 msgr2=0x55824f35bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.579+0000 7f309b400700 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f35bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.579+0000 7f30a0c00700 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f35bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.580+0000 7f30a0c00700 1 -- 172.21.15.90:0/2368940407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824f359e00 con 0x55824f3587d0 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.580+0000 7f30a0c00700 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f309402ad00 tx=0x7f309402bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fbda5a0989560e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.580+0000 7f3090c00700 1 -- 172.21.15.90:0/2368940407 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940217b0 con 0x55824f3587d0 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.580+0000 7f3090c00700 1 -- 172.21.15.90:0/2368940407 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3094005bc0 con 0x55824f3587d0 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.580+0000 7f3090c00700 1 -- 172.21.15.90:0/2368940407 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094022d20 con 0x55824f3587d0 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 -- 172.21.15.90:0/2368940407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 msgr2=0x55824f346e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f309402ad00 tx=0x7f309402bf80 comp rx=0 tx=0).stop 2024-06-02T15:49:34.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 -- 172.21.15.90:0/2368940407 shutdown_connections 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f346e50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f352880 0x55824f352260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/2368940407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f357eb0 0x55824f35bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 -- 172.21.15.90:0/2368940407 >> 172.21.15.90:0/2368940407 conn(0x55824f280be0 msgr2=0x55824f358bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.582+0000 7f30ad4beb80 1 -- 172.21.15.90:0/2368940407 shutdown_connections 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.583+0000 7f30ad4beb80 1 -- 172.21.15.90:0/2368940407 wait complete. 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.583+0000 7f30ad4beb80 1 -- start start 2024-06-02T15:49:34.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.583+0000 7f30ad4beb80 1 Processor -- start 2024-06-02T15:49:34.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.584+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f364d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.584+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f357eb0 0x55824f3841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.584+0000 7f30ad4beb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f309b400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f357eb0 0x55824f3841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1ab8f0 con 0x55824f352880 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1acd60 con 0x55824f3587d0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30ad4beb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1abfc0 con 0x55824f357eb0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30a0c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33014/0 (socket says 172.21.15.90:33014) 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f30a0c00700 1 -- 172.21.15.90:0/4205709754 learned_addr learned my addr 172.21.15.90:0/4205709754 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.585+0000 7f309be00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f364d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30ad4beb80 10 client.0 _ll_register_callbacks cb 0x55824f24ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30a0c00700 1 -- 172.21.15.90:0/4205709754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f357eb0 msgr2=0x55824f3841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30a0c00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f357eb0 0x55824f3841e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30a0c00700 1 -- 172.21.15.90:0/4205709754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 msgr2=0x55824f364d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30a0c00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f364d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f30a0c00700 1 -- 172.21.15.90:0/4205709754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824f3e6680 con 0x55824f3587d0 2024-06-02T15:49:34.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.586+0000 7f309be00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f364d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:34.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.587+0000 7f30a0c00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3094017040 tx=0x7f3094009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc883ee9b796bc38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:34.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.587+0000 7f3086000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.587+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309402a630 con 0x55824f3587d0 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.587+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f309402a7d0 con 0x55824f3587d0 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.587+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094005bc0 con 0x55824f3587d0 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.588+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f309404e050 con 0x55824f3587d0 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.589+0000 7f30ad4beb80 10 client.25892 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:34.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.589+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55824f3e7830 con 0x55824f3587d0 2024-06-02T15:49:34.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.590+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3094005bc0 con 0x55824f3587d0 2024-06-02T15:49:34.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f309403c020 con 0x55824f3587d0 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 10 client.25892 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 20 client.25892 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f3e7e30 0x55824f3ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 4 client.25892 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55824f2ac570 con 0x55824f3e7e30 2024-06-02T15:49:34.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.591+0000 7f309be00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f3e7e30 0x55824f3ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:34.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.592+0000 7f309be00700 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f3e7e30 0x55824f3ea250 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2229da074b589c server_cookie=8a9cb3d3c1538f5c in_seq=0 out_seq=0 2024-06-02T15:49:34.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.592+0000 7f3086000700 10 client.25892 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:34.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.592+0000 7f3086000700 1 -- 172.21.15.90:0/4205709754 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55824f2ac570 con 0x55824f3e7e30 2024-06-02T15:49:34.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.592+0000 7f3086000700 10 client.25892 handle_command_reply: tid=0 2024-06-02T15:49:34.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.592+0000 7f30ad4beb80 1 client.25892 shutdown 2024-06-02T15:49:34.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.593+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 msgr2=0x55824f3848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.593+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3094017040 tx=0x7f3094009660 comp rx=0 tx=0).stop 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 shutdown_connections 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55824f3e7e30 0x55824f3ea250 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f3086000700 0 client.25892 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55824f3587d0 0x55824f3848d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55824f352880 0x55824f364d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 --2- 172.21.15.90:0/4205709754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55824f357eb0 0x55824f3841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 >> 172.21.15.90:0/4205709754 conn(0x55824f280be0 msgr2=0x55824f35a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.594+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 shutdown_connections 2024-06-02T15:49:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.595+0000 7f30ad4beb80 1 -- 172.21.15.90:0/4205709754 wait complete. 2024-06-02T15:49:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.595+0000 7f30ad4beb80 20 client.25892 trim_cache size 0 max 16384 2024-06-02T15:49:34.594 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "9744a362-141c-4314-b3c3-3115ac9f31b3": { 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "9744a362-141c-4314-b3c3-3115ac9f31b3", 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:49:34.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 msgr2=0x7f30800b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 0x7f30800b4e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3094022000 tx=0x7f3094024000 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 msgr2=0x7f309c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f308800e9a0 tx=0x7f308800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 shutdown_connections 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30800b2940 0x7f30800b4e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f309c108500 0x7f309c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f309c1035f0 0x7f309c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 --2- 172.21.15.90:0/712383323 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f309c103f90 0x7f309c19c3f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.597+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 >> 172.21.15.90:0/712383323 conn(0x7f309c07a6a0 msgr2=0x7f309c101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.598+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 shutdown_connections 2024-06-02T15:49:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:49:34.598+0000 7f30a2c00700 1 -- 172.21.15.90:0/712383323 wait complete. 2024-06-02T15:49:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 -- 172.21.15.90:0/2000652141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c1028e0 msgr2=0x7fe52c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 --2- 172.21.15.90:0/2000652141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c1028e0 0x7fe52c102d00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 -- 172.21.15.90:0/2000652141 shutdown_connections 2024-06-02T15:49:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 --2- 172.21.15.90:0/2000652141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe52c104490 0x7fe52c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 --2- 172.21.15.90:0/2000652141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c103ad0 0x7fe52c103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 --2- 172.21.15.90:0/2000652141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c1028e0 0x7fe52c102d00 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.113+0000 7fe530e00640 1 -- 172.21.15.90:0/2000652141 >> 172.21.15.90:0/2000652141 conn(0x7fe52c0fde50 msgr2=0x7fe52c100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.115+0000 7fe530e00640 1 -- 172.21.15.90:0/2000652141 shutdown_connections 2024-06-02T15:49:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.115+0000 7fe530e00640 1 -- 172.21.15.90:0/2000652141 wait complete. 2024-06-02T15:49:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.116+0000 7fe530e00640 1 Processor -- start 2024-06-02T15:49:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.116+0000 7fe530e00640 1 -- start start 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe52c103ad0 0x7fe52c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c1ad330 0x7fe52c1af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e430 con 0x7fe52c103ad0 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e2b0 con 0x7fe52c104490 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe530e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c10e5b0 con 0x7fe52c1ad330 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33024/0 (socket says 172.21.15.90:33024) 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 -- 172.21.15.90:0/191721761 learned_addr learned my addr 172.21.15.90:0/191721761 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52be00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c1ad330 0x7fe52c1af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 -- 172.21.15.90:0/191721761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c1ad330 msgr2=0x7fe52c1af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c1ad330 0x7fe52c1af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 -- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe52c103ad0 msgr2=0x7fe52c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe52c103ad0 0x7fe52c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.117+0000 7fe52aa00640 1 -- 172.21.15.90:0/191721761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c1afc60 con 0x7fe52c104490 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe52aa00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe51800ea60 tx=0x7fe51800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe523e00640 1 -- 172.21.15.90:0/191721761 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800cde0 con 0x7fe52c104490 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe523e00640 1 -- 172.21.15.90:0/191721761 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5180035f0 con 0x7fe52c104490 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe523e00640 1 -- 172.21.15.90:0/191721761 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518010690 con 0x7fe52c104490 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c1afe90 con 0x7fe52c104490 2024-06-02T15:49:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.118+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1b0300 con 0x7fe52c104490 2024-06-02T15:49:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.123+0000 7fe535235740 1 Processor -- start 2024-06-02T15:49:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.123+0000 7fe523e00640 1 -- 172.21.15.90:0/191721761 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe5180037a0 con 0x7fe52c104490 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.123+0000 7fe535235740 1 -- start start 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740ba3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740baace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563740b12b90 con 0x563740ba2cf0 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563740b02730 con 0x563740ba36e0 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe535235740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563740b06770 con 0x563740ba8850 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe52b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe52b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33040/0 (socket says 172.21.15.90:33040) 2024-06-02T15:49:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe52b400640 1 -- 172.21.15.90:0/667091233 learned_addr learned my addr 172.21.15.90:0/667091233 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.124+0000 7fe52be00640 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740ba3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 -- 172.21.15.90:0/667091233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 msgr2=0x563740baace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740baace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 -- 172.21.15.90:0/667091233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 msgr2=0x563740ba3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740ba3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 -- 172.21.15.90:0/667091233 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563740a9b8b0 con 0x563740ba36e0 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52be00640 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740ba3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.125+0000 7fe52b400640 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe51c031500 tx=0x7fe51c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453d9b148a2b9f0c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe523e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 0x7fe5100b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe520c00640 1 -- 172.21.15.90:0/667091233 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c03e070 con 0x563740ba36e0 2024-06-02T15:49:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe520c00640 1 -- 172.21.15.90:0/667091233 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51c0053c0 con 0x563740ba36e0 2024-06-02T15:49:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe520c00640 1 -- 172.21.15.90:0/667091233 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c035040 con 0x563740ba36e0 2024-06-02T15:49:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe523e00640 1 -- 172.21.15.90:0/191721761 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe5180e2990 con 0x7fe52c104490 2024-06-02T15:49:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.126+0000 7fe52be00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 0x7fe5100b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- 172.21.15.90:0/667091233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 msgr2=0x563740ba81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe51c031500 tx=0x7fe51c009650 comp rx=0 tx=0).stop 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- 172.21.15.90:0/667091233 shutdown_connections 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740baace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba81e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 --2- 172.21.15.90:0/667091233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740ba3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- 172.21.15.90:0/667091233 >> 172.21.15.90:0/667091233 conn(0x563740b9e360 msgr2=0x563740b9e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- 172.21.15.90:0/667091233 shutdown_connections 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe52be00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 0x7fe5100b4ca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563740ba3f40 tx=0x7fe524009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- 172.21.15.90:0/667091233 wait complete. 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.127+0000 7fe535235740 1 -- start start 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 Processor -- start 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740bad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5637407e8790 con 0x563740ba2cf0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563740aaabc0 con 0x563740ba36e0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563740bab7a0 con 0x563740ba8850 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.128+0000 7fe535235740 10 client.0 _ll_register_callbacks cb 0x563740a88400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45600/0 (socket says 172.21.15.90:45600) 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 -- 172.21.15.90:0/1145116324 learned_addr learned my addr 172.21.15.90:0/1145116324 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52aa00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740bad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52be00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 -- 172.21.15.90:0/1145116324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 msgr2=0x563740bb4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 -- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 msgr2=0x563740bad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740bad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52b400640 1 -- 172.21.15.90:0/1145116324 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563740817210 con 0x563740ba2cf0 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52aa00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740bad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.129+0000 7fe52be00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.130+0000 7fe52b400640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b5749eeb247fb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.130+0000 7fe515600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.130+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c04a020 con 0x563740ba2cf0 2024-06-02T15:49:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.130+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c0053c0 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.130+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe51c031a20 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe51c055050 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 10 client.15741 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637408e3620 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe51c0053c0 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe51c0035f0 con 0x563740ba2cf0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 10 client.15741 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 20 client.15741 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740bb81c0 0x563740bba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 4 client.15741 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.131+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563740b14ae0 con 0x563740bb81c0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.132+0000 7fe52aa00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740bb81c0 0x563740bba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.132+0000 7fe52aa00640 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740bb81c0 0x563740bba5b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8942733ab1739c server_cookie=5f165354e919fdb4 in_seq=0 out_seq=0 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.132+0000 7fe515600640 10 client.15741 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.133+0000 7fe515600640 1 -- 172.21.15.90:0/1145116324 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563740b14ae0 con 0x563740bb81c0 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.133+0000 7fe515600640 10 client.15741 handle_command_reply: tid=0 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.133+0000 7fe535235740 1 client.15741 shutdown 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 msgr2=0x563740bacae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c004270 comp rx=0 tx=0).stop 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 shutdown_connections 2024-06-02T15:49:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740bb81c0 0x563740bba5b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740bad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 --2- 172.21.15.90:0/1145116324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba2cf0 0x563740bacae0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.135+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 >> 172.21.15.90:0/1145116324 conn(0x563740b9e360 msgr2=0x563740b9d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.136+0000 7fe515600640 0 client.15741 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.136+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 shutdown_connections 2024-06-02T15:49:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.136+0000 7fe535235740 1 -- 172.21.15.90:0/1145116324 wait complete. 2024-06-02T15:49:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.136+0000 7fe535235740 20 client.15741 trim_cache size 0 max 16384 2024-06-02T15:49:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.154+0000 7fe535235740 1 Processor -- start 2024-06-02T15:49:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.154+0000 7fe535235740 1 -- start start 2024-06-02T15:49:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.154+0000 7fe535235740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.154+0000 7fe535235740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe535235740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 0x563740bac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe535235740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1bd190 con 0x563740ba1860 2024-06-02T15:49:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe535235740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1baa60 con 0x563740ba36e0 2024-06-02T15:49:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe535235740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1b88b0 con 0x563740ba8850 2024-06-02T15:49:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:43776/0 (socket says 172.21.15.90:43776) 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe52aa00640 1 -- 172.21.15.90:0/3350335635 learned_addr learned my addr 172.21.15.90:0/3350335635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.155+0000 7fe52be00640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 0x563740bac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52b400640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 -- 172.21.15.90:0/3350335635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 msgr2=0x563740ba1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 -- 172.21.15.90:0/3350335635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 msgr2=0x563740bac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 0x563740bac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 -- 172.21.15.90:0/3350335635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563740bbabe0 con 0x563740ba8850 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52be00640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 0x563740bac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52b400640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.156+0000 7fe52aa00640 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5180a6e70 tx=0x7fe518004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22174172c13728dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.157+0000 7fe517e00640 1 -- 172.21.15.90:0/3350335635 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518091040 con 0x563740ba8850 2024-06-02T15:49:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.157+0000 7fe517e00640 1 -- 172.21.15.90:0/3350335635 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5180a9710 con 0x563740ba8850 2024-06-02T15:49:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.157+0000 7fe517e00640 1 -- 172.21.15.90:0/3350335635 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180a8e60 con 0x563740ba8850 2024-06-02T15:49:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 -- 172.21.15.90:0/3350335635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 msgr2=0x563740bb4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5180a6e70 tx=0x7fe518004560 comp rx=0 tx=0).stop 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 -- 172.21.15.90:0/3350335635 shutdown_connections 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba1860 0x563740bac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb4020 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 --2- 172.21.15.90:0/3350335635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba36e0 0x563740ba1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 -- 172.21.15.90:0/3350335635 >> 172.21.15.90:0/3350335635 conn(0x563740b9e360 msgr2=0x563740ba7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 -- 172.21.15.90:0/3350335635 shutdown_connections 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.158+0000 7fe535235740 1 -- 172.21.15.90:0/3350335635 wait complete. 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.159+0000 7fe535235740 1 -- start start 2024-06-02T15:49:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.159+0000 7fe535235740 1 Processor -- start 2024-06-02T15:49:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.159+0000 7fe535235740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.159+0000 7fe535235740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 0x563740bb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.159+0000 7fe535235740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe535235740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1b8f20 con 0x563740ba36e0 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe535235740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1ba390 con 0x563740ba1860 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe535235740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c1b95f0 con 0x563740ba8850 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe535235740 10 client.0 _ll_register_callbacks cb 0x7fe518005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 0x563740bb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:33068/0 (socket says 172.21.15.90:33068) 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 -- 172.21.15.90:0/3557175636 learned_addr learned my addr 172.21.15.90:0/3557175636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 0x563740bb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45612/0 (socket says 172.21.15.90:45612) 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52be00640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 -- 172.21.15.90:0/3557175636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 msgr2=0x563740bb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 -- 172.21.15.90:0/3557175636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 msgr2=0x563740bb1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 0x563740bb1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52b400640 1 -- 172.21.15.90:0/3557175636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563740bb2c80 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.160+0000 7fe52be00640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe52b400640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe51c003bb0 tx=0x7fe51c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df1f91f7a6e8891 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe515600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c05e060 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe51c005480 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5180a9710 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe51c065050 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe535235740 10 client.25907 fetch_fsmap learned FSMap version 31 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.161+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563740bb3f00 con 0x563740ba1860 2024-06-02T15:49:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.162+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5180a9710 con 0x563740ba1860 2024-06-02T15:49:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.162+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fe51c0046c0 con 0x563740ba1860 2024-06-02T15:49:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.162+0000 7fe535235740 10 client.25907 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T15:49:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.162+0000 7fe535235740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.163+0000 7fe535235740 20 client.25907 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.163+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740c23010 0x563740c25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.163+0000 7fe535235740 4 client.25907 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.163+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563740b0e2c0 con 0x563740c23010 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.163+0000 7fe52aa00640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740c23010 0x563740c25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.164+0000 7fe515600640 10 client.25907 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:35.164+0000 7fe52aa00640 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740c23010 0x563740c25400 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d620509871f393 server_cookie=4c3cc04a1ff6c830 in_seq=0 out_seq=0 2024-06-02T15:49:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:35 smithi120 ceph-mon[36451]: pgmap v668: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.27k op/s 2024-06-02T15:49:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:35 smithi135 ceph-mon[36839]: pgmap v668: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.27k op/s 2024-06-02T15:49:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:35 smithi090 ceph-mon[31595]: pgmap v668: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 1.27k op/s 2024-06-02T15:49:35.812 INFO:tasks.cephfs.filesystem:scrub status for tag:9744a362-141c-4314-b3c3-3115ac9f31b3 - {'path': '/', 'tag': '9744a362-141c-4314-b3c3-3115ac9f31b3', 'options': 'recursive,force'} 2024-06-02T15:49:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:37 smithi090 ceph-mon[31595]: pgmap v669: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 938 op/s 2024-06-02T15:49:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:37 smithi120 ceph-mon[36451]: pgmap v669: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 938 op/s 2024-06-02T15:49:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:37 smithi135 ceph-mon[36839]: pgmap v669: 129 pgs: 129 active+clean; 26 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 938 op/s 2024-06-02T15:49:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.424+0000 7fe515600640 1 -- 172.21.15.90:0/3557175636 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563740b0e2c0 con 0x563740c23010 2024-06-02T15:49:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.424+0000 7fe515600640 10 client.25907 handle_command_reply: tid=0 2024-06-02T15:49:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.424+0000 7fe535235740 1 client.25907 shutdown 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 msgr2=0x563740bb1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe51c003bb0 tx=0x7fe51c0038a0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 shutdown_connections 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563740c23010 0x563740c25400 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563740ba8850 0x563740bb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563740ba36e0 0x563740bb1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 --2- 172.21.15.90:0/3557175636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563740ba1860 0x563740bb1140 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 >> 172.21.15.90:0/3557175636 conn(0x563740b9e360 msgr2=0x563740ba3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.425+0000 7fe515600640 0 client.25907 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.426+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 shutdown_connections 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.426+0000 7fe535235740 1 -- 172.21.15.90:0/3557175636 wait complete. 2024-06-02T15:49:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.426+0000 7fe535235740 20 client.25907 trim_cache size 0 max 16384 2024-06-02T15:49:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:49:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:49:38.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:49:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.428+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 msgr2=0x7fe5100b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.428+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 0x7fe5100b4ca0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563740ba3f40 tx=0x7fe524009500 comp rx=0 tx=0).stop 2024-06-02T15:49:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.428+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 msgr2=0x7fe52c197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.428+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe51800ea60 tx=0x7fe51800ef20 comp rx=0 tx=0).stop 2024-06-02T15:49:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 shutdown_connections 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe5100b27e0 0x7fe5100b4ca0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe52c1ad330 0x7fe52c1af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe52c104490 0x7fe52c197090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 --2- 172.21.15.90:0/191721761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe52c103ad0 0x7fe52c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 >> 172.21.15.90:0/191721761 conn(0x7fe52c0fde50 msgr2=0x7fe52c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 shutdown_connections 2024-06-02T15:49:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:38.429+0000 7fe530e00640 1 -- 172.21.15.90:0/191721761 wait complete. 2024-06-02T15:49:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:39 smithi120 ceph-mon[36451]: pgmap v670: 129 pgs: 129 active+clean; 26 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2024-06-02T15:49:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:39 smithi135 ceph-mon[36839]: pgmap v670: 129 pgs: 129 active+clean; 26 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2024-06-02T15:49:39.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:39 smithi090 ceph-mon[31595]: pgmap v670: 129 pgs: 129 active+clean; 26 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2024-06-02T15:49:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:41 smithi120 ceph-mon[36451]: pgmap v671: 129 pgs: 129 active+clean; 30 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 625 op/s 2024-06-02T15:49:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:41 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:41 smithi135 ceph-mon[36839]: pgmap v671: 129 pgs: 129 active+clean; 30 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 625 op/s 2024-06-02T15:49:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:41 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:41.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:41 smithi090 ceph-mon[31595]: pgmap v671: 129 pgs: 129 active+clean; 30 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 625 op/s 2024-06-02T15:49:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:41 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:43 smithi120 ceph-mon[36451]: pgmap v672: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-06-02T15:49:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:43 smithi135 ceph-mon[36839]: pgmap v672: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-06-02T15:49:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:43 smithi090 ceph-mon[31595]: pgmap v672: 129 pgs: 129 active+clean; 32 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-06-02T15:49:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: pgmap v673: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 750 op/s 2024-06-02T15:49:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:45.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:45 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: pgmap v673: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 750 op/s 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.595 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.597 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.598 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.599 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.600 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.601 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.602 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.603 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.604 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.605 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.606 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.607 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.608 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.609 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.610 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.610 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:45.610 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:45 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: pgmap v673: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 750 op/s 2024-06-02T15:49:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:45.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:45 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:46.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:46 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:46.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:46 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:46 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:49:47.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:47 smithi120 ceph-mon[36451]: pgmap v674: 129 pgs: 129 active+clean; 34 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 766 op/s 2024-06-02T15:49:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:47 smithi135 ceph-mon[36839]: pgmap v674: 129 pgs: 129 active+clean; 34 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 766 op/s 2024-06-02T15:49:47.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:47 smithi090 ceph-mon[31595]: pgmap v674: 129 pgs: 129 active+clean; 34 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 766 op/s 2024-06-02T15:49:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:49 smithi120 ceph-mon[36451]: pgmap v675: 129 pgs: 129 active+clean; 34 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 714 op/s 2024-06-02T15:49:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:49 smithi135 ceph-mon[36839]: pgmap v675: 129 pgs: 129 active+clean; 34 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 714 op/s 2024-06-02T15:49:49.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:49 smithi090 ceph-mon[31595]: pgmap v675: 129 pgs: 129 active+clean; 34 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 714 op/s 2024-06-02T15:49:50.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:50.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:50 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:50.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:50 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:50 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.577 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.578 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.579 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.580 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.581 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.582 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.583 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.584 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.585 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.586 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.587 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.588 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.589 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.590 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.591 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.592 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.593 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.594 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.595 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.596 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.598 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.599 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.600 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.601 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.602 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.603 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.604 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.605 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.606 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.607 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.608 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: pgmap v676: 129 pgs: 129 active+clean; 41 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 695 op/s 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.610 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.611 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.612 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.613 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.614 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.615 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.616 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.617 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.618 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.619 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.621 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.622 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.623 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.624 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.625 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.626 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.627 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.628 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.629 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:51.630 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:51 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.632 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.633 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.634 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.635 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.637 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.638 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.639 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.641 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.642 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.643 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.645 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.647 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.648 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.649 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.650 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.651 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.652 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.653 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.654 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.655 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.657 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.658 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.659 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.660 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.661 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.662 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.663 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.666 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.667 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.668 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.669 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.670 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.671 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.672 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.674 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.675 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.676 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.677 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.678 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: pgmap v676: 129 pgs: 129 active+clean; 41 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 695 op/s 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.679 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.680 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.681 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.682 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.683 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.684 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.685 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.686 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.688 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.689 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.690 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.692 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.693 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.694 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.696 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.697 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.698 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.699 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:51.700 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:51 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 -- 172.21.15.90:0/3023340758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d0075430 msgr2=0x7fb2d0075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3023340758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d0075430 0x7fb2d0075890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009860 tx=0x7fb2bc02a3d0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 -- 172.21.15.90:0/3023340758 shutdown_connections 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3023340758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d010a960 0x7fb2d010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3023340758 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d0075430 0x7fb2d0075890 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3023340758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0076a10 0x7fb2d0074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.604+0000 7fb2d5000640 1 -- 172.21.15.90:0/3023340758 >> 172.21.15.90:0/3023340758 conn(0x7fb2d00fe040 msgr2=0x7fb2d0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.605+0000 7fb2d5000640 1 -- 172.21.15.90:0/3023340758 shutdown_connections 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.605+0000 7fb2d5000640 1 -- 172.21.15.90:0/3023340758 wait complete. 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.606+0000 7fb2d5000640 1 Processor -- start 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.607+0000 7fb2d5000640 1 -- start start 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d0076a10 0x7fb2d006ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d010a960 0x7fb2d0073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0103590 con 0x7fb2d0076a10 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0103410 con 0x7fb2d010a960 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2d5000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0103710 con 0x7fb2d0075430 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d0076a10 0x7fb2d006ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2ce800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d010a960 0x7fb2d0073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:43684/0 (socket says 172.21.15.90:43684) 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.608+0000 7fb2cde00640 1 -- 172.21.15.90:0/3240817833 learned_addr learned my addr 172.21.15.90:0/3240817833 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.609+0000 7fb2cde00640 1 -- 172.21.15.90:0/3240817833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d010a960 msgr2=0x7fb2d0073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.609+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d010a960 0x7fb2d0073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.609+0000 7fb2cde00640 1 -- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d0076a10 msgr2=0x7fb2d006ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.609+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d0076a10 0x7fb2d006ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.609+0000 7fb2cde00640 1 -- 172.21.15.90:0/3240817833 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d00745e0 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.610+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d950 tx=0x7fb2c000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.611+0000 7fb2c6a00640 1 -- 172.21.15.90:0/3240817833 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0017030 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.611+0000 7fb2c6a00640 1 -- 172.21.15.90:0/3240817833 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c0004020 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.611+0000 7fb2c6a00640 1 -- 172.21.15.90:0/3240817833 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00047a0 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.611+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d00748d0 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.611+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01afe80 con 0x7fb2d0075430 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.616+0000 7fb2d7e72740 1 Processor -- start 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.616+0000 7fb2d7e72740 1 -- start start 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6d520 0x563df5b72020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b72690 0x563df5b74b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563df5adcb90 con 0x563df5b6d520 2024-06-02T15:49:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563df5acc730 con 0x563df5b6cb30 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563df5ad0770 con 0x563df5b72690 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59272/0 (socket says 172.21.15.90:59272) 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 -- 172.21.15.90:0/2175856654 learned_addr learned my addr 172.21.15.90:0/2175856654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2cd400640 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b72690 0x563df5b74b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 -- 172.21.15.90:0/2175856654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b72690 msgr2=0x563df5b74b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b72690 0x563df5b74b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.617+0000 7fb2ce800640 1 -- 172.21.15.90:0/2175856654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6d520 msgr2=0x563df5b72020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.618+0000 7fb2ce800640 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6d520 0x563df5b72020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.618+0000 7fb2ce800640 1 -- 172.21.15.90:0/2175856654 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df5a658b0 con 0x563df5b6cb30 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.618+0000 7fb2ce800640 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800e480 tx=0x7fb2c80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b167d1eb69da47 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.618+0000 7fb2bb400640 1 -- 172.21.15.90:0/2175856654 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003970 con 0x563df5b6cb30 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.619+0000 7fb2bb400640 1 -- 172.21.15.90:0/2175856654 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c800d7d0 con 0x563df5b6cb30 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.619+0000 7fb2bb400640 1 -- 172.21.15.90:0/2175856654 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80044c0 con 0x563df5b6cb30 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.619+0000 7fb2d7e72740 1 -- 172.21.15.90:0/2175856654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 msgr2=0x563df5b6cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.619+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800e480 tx=0x7fb2c80095c0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.620+0000 7fb2d7e72740 1 -- 172.21.15.90:0/2175856654 shutdown_connections 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.620+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b72690 0x563df5b74b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.620+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6d520 0x563df5b72020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.620+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/2175856654 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6cf50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.620+0000 7fb2d7e72740 1 -- 172.21.15.90:0/2175856654 >> 172.21.15.90:0/2175856654 conn(0x563df5b681a0 msgr2=0x563df5b685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.621+0000 7fb2c6a00640 1 -- 172.21.15.90:0/3240817833 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb2c0004940 con 0x7fb2d0075430 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.621+0000 7fb2d7e72740 1 -- 172.21.15.90:0/2175856654 shutdown_connections 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.622+0000 7fb2d7e72740 1 -- 172.21.15.90:0/2175856654 wait complete. 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.622+0000 7fb2d7e72740 1 -- start start 2024-06-02T15:49:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.623+0000 7fb2d7e72740 1 Processor -- start 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.623+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6cb30 0x563df5b76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563df57b2790 con 0x563df5b6cb30 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563df5a74bc0 con 0x563df5b72690 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563df5b75640 con 0x563df5b7c030 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2d7e72740 10 client.0 _ll_register_callbacks cb 0x563df5a52400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2c6a00640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 0x7fb2b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59274/0 (socket says 172.21.15.90:59274) 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2cde00640 1 -- 172.21.15.90:0/1207411954 learned_addr learned my addr 172.21.15.90:0/1207411954 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2c6a00640 1 -- 172.21.15.90:0/3240817833 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c00e1f10 con 0x7fb2d0075430 2024-06-02T15:49:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.624+0000 7fb2cd400640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b7e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 -- 172.21.15.90:0/1207411954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 msgr2=0x563df5b7e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b7e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 -- 172.21.15.90:0/1207411954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6cb30 msgr2=0x563df5b76b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6cb30 0x563df5b76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 -- 172.21.15.90:0/1207411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df57e1210 con 0x563df5b72690 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2ce800640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 0x7fb2b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2cde00640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005210 tx=0x7fb2c00a8fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad0795ca65daaab6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2afe00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a9820 con 0x563df5b72690 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c00041d0 con 0x563df5b72690 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.625+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c800d7d0 con 0x563df5b72690 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.626+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00a9120 con 0x563df5b72690 2024-06-02T15:49:51.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.626+0000 7fb2ce800640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 0x7fb2b40b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009ba0 tx=0x7fb2c8005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.629+0000 7fb2d7e72740 10 client.25919 fetch_fsmap learned FSMap version 33 2024-06-02T15:49:51.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.629+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563df58ad620 con 0x563df5b72690 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.629+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c800d7d0 con 0x563df5b72690 2024-06-02T15:49:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c00ad120 con 0x563df5b72690 2024-06-02T15:49:51.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.725 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 10 client.25919 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 20 client.25919 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5b82a30 0x563df5b84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 4 client.25919 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df5adeae0 con 0x563df5b82a30 2024-06-02T15:49:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.632+0000 7fb2cd400640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5b82a30 0x563df5b84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: pgmap v676: 129 pgs: 129 active+clean; 41 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 695 op/s 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.731 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.633+0000 7fb2cd400640 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5b82a30 0x563df5b84e20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2e6526d930060f server_cookie=60a1067054d8d7bc in_seq=0 out_seq=0 2024-06-02T15:49:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.633+0000 7fb2afe00640 10 client.25919 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.634+0000 7fb2afe00640 1 -- 172.21.15.90:0/1207411954 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563df5adeae0 con 0x563df5b82a30 2024-06-02T15:49:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.634+0000 7fb2afe00640 10 client.25919 handle_command_reply: tid=0 2024-06-02T15:49:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.634+0000 7fb2d7e72740 1 client.25919 shutdown 2024-06-02T15:49:51.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.734 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 msgr2=0x563df5b77070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005210 tx=0x7fb2c00a8fe0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 shutdown_connections 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5b82a30 0x563df5b84e20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b7e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b72690 0x563df5b77070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/1207411954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b6cb30 0x563df5b76b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 >> 172.21.15.90:0/1207411954 conn(0x563df5b681a0 msgr2=0x563df5b67310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2afe00640 0 client.25919 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 shutdown_connections 2024-06-02T15:49:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.635+0000 7fb2d7e72740 1 -- 172.21.15.90:0/1207411954 wait complete. 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.638+0000 7fb2d7e72740 20 client.25919 trim_cache size 0 max 16384 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:51 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.671+0000 7fb2d7e72740 1 Processor -- start 2024-06-02T15:49:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.671+0000 7fb2d7e72740 1 -- start start 2024-06-02T15:49:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b76c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0012640 con 0x563df5b72690 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000feb0 con 0x563df5b6cb30 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000dd00 con 0x563df5b7c030 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59280/0 (socket says 172.21.15.90:59280) 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2cd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2cde00640 1 -- 172.21.15.90:0/623993316 learned_addr learned my addr 172.21.15.90:0/623993316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.672+0000 7fb2ce800640 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b76c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cde00640 1 -- 172.21.15.90:0/623993316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 msgr2=0x563df5b76c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cde00640 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b76c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cde00640 1 -- 172.21.15.90:0/623993316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 msgr2=0x563df5b76740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cde00640 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b76740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cde00640 1 -- 172.21.15.90:0/623993316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df5b85480 con 0x563df5b6cb30 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.673+0000 7fb2cd400640 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.674+0000 7fb2cde00640 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00a51a0 tx=0x7fb2c00097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e238cf2cf673628b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.674+0000 7fb2baa00640 1 -- 172.21.15.90:0/623993316 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a9820 con 0x563df5b6cb30 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.674+0000 7fb2baa00640 1 -- 172.21.15.90:0/623993316 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c00a5710 con 0x563df5b6cb30 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.674+0000 7fb2baa00640 1 -- 172.21.15.90:0/623993316 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00df160 con 0x563df5b6cb30 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.675+0000 7fb2d7e72740 1 -- 172.21.15.90:0/623993316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 msgr2=0x563df5b6ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.675+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00a51a0 tx=0x7fb2c00097b0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 -- 172.21.15.90:0/623993316 shutdown_connections 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b7c030 0x563df5b76c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/623993316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b6cb30 0x563df5b6ba20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 -- 172.21.15.90:0/623993316 >> 172.21.15.90:0/623993316 conn(0x563df5b681a0 msgr2=0x563df5b68580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 -- 172.21.15.90:0/623993316 shutdown_connections 2024-06-02T15:49:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 -- 172.21.15.90:0/623993316 wait complete. 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 -- start start 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.676+0000 7fb2d7e72740 1 Processor -- start 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b94b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b7c030 0x563df5b95b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000e370 con 0x563df5b72690 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000f7e0 con 0x563df5b7c030 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000ea40 con 0x563df5b6cb30 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2d7e72740 10 client.0 _ll_register_callbacks cb 0x7fb2c00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2cde00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:43712/0 (socket says 172.21.15.90:43712) 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.677+0000 7fb2cde00640 1 -- 172.21.15.90:0/3925861802 learned_addr learned my addr 172.21.15.90:0/3925861802 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2ce800640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b7c030 0x563df5b95b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 -- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b7c030 msgr2=0x563df5b95b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b7c030 0x563df5b95b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cd400640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b94b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 -- 172.21.15.90:0/3925861802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 msgr2=0x563df5b94b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b94b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 -- 172.21.15.90:0/3925861802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df5b812a0 con 0x563df5b6cb30 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cd400640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b94b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2cde00640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00c0040 tx=0x7fb2c00e2190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a186fdb244da1e54 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.678+0000 7fb2afe00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a9820 con 0x563df5b6cb30 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb2c00041d0 con 0x563df5b6cb30 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c00a5710 con 0x563df5b6cb30 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00ca0d0 con 0x563df5b6cb30 2024-06-02T15:49:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2d7e72740 10 client.25876 fetch_fsmap learned FSMap version 33 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.679+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563df5bed530 con 0x563df5b6cb30 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.680+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c00a5710 con 0x563df5b6cb30 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c00b6080 con 0x563df5b6cb30 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 10 client.25876 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 20 client.25876 populate_metadata read hostname 'smithi090' 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5beddc0 0x563df5bf01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 4 client.25876 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563df5ad82c0 con 0x563df5beddc0 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.681+0000 7fb2cd400640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5beddc0 0x563df5bf01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.682+0000 7fb2cd400640 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5beddc0 0x563df5bf01b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1521e4e4742b7dd server_cookie=2db05296ef3bd6d0 in_seq=0 out_seq=0 2024-06-02T15:49:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:51.682+0000 7fb2afe00640 10 client.25876 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.270 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.271 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.272 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.273 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.274 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.275 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:49:52.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:52 smithi120 ceph-mon[36451]: pgmap v677: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 453 op/s 2024-06-02T15:49:52.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:52 smithi135 ceph-mon[36839]: pgmap v677: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 453 op/s 2024-06-02T15:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:49:52.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:49:52.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:52 smithi090 ceph-mon[31595]: pgmap v677: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 453 op/s 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.969+0000 7fb2afe00640 1 -- 172.21.15.90:0/3925861802 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563df5ad82c0 con 0x563df5beddc0 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.969+0000 7fb2afe00640 10 client.25876 handle_command_reply: tid=0 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.969+0000 7fb2d7e72740 1 client.25876 shutdown 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 msgr2=0x563df5b945f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00c0040 tx=0x7fb2c00e2190 comp rx=0 tx=0).stop 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 shutdown_connections 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563df5beddc0 0x563df5bf01b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563df5b7c030 0x563df5b95b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2afe00640 0 client.25876 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:49:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563df5b72690 0x563df5b94b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 --2- 172.21.15.90:0/3925861802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563df5b6cb30 0x563df5b945f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 >> 172.21.15.90:0/3925861802 conn(0x563df5b681a0 msgr2=0x563df5b76740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 shutdown_connections 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 1 -- 172.21.15.90:0/3925861802 wait complete. 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.970+0000 7fb2d7e72740 20 client.25876 trim_cache size 0 max 16384 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:49:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:49:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 msgr2=0x7fb2b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 0x7fb2b40b4dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009ba0 tx=0x7fb2c8005bd0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 msgr2=0x7fb2d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:49:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d950 tx=0x7fb2c000de10 comp rx=0 tx=0).stop 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 shutdown_connections 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb2b40b2900 0x7fb2b40b4dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2d010a960 0x7fb2d0073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d0076a10 0x7fb2d006ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 --2- 172.21.15.90:0/3240817833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2d0075430 0x7fb2d006ea20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.972+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 >> 172.21.15.90:0/3240817833 conn(0x7fb2d00fe040 msgr2=0x7fb2d0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.973+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 shutdown_connections 2024-06-02T15:49:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:49:54.973+0000 7fb2d5000640 1 -- 172.21.15.90:0/3240817833 wait complete. 2024-06-02T15:49:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:55 smithi120 ceph-mon[36451]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-06-02T15:49:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:55 smithi135 ceph-mon[36839]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-06-02T15:49:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:55 smithi090 ceph-mon[31595]: pgmap v678: 129 pgs: 129 active+clean; 51 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-06-02T15:49:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:57 smithi120 ceph-mon[36451]: pgmap v679: 129 pgs: 129 active+clean; 44 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 1.03k op/s 2024-06-02T15:49:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:57 smithi135 ceph-mon[36839]: pgmap v679: 129 pgs: 129 active+clean; 44 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 1.03k op/s 2024-06-02T15:49:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:57 smithi090 ceph-mon[31595]: pgmap v679: 129 pgs: 129 active+clean; 44 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 1.03k op/s 2024-06-02T15:49:59.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:49:59 smithi090 ceph-mon[31595]: pgmap v680: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 1.36k op/s 2024-06-02T15:49:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:49:59 smithi120 ceph-mon[36451]: pgmap v680: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 1.36k op/s 2024-06-02T15:49:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:49:59 smithi135 ceph-mon[36839]: pgmap v680: 129 pgs: 129 active+clean; 48 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 1.36k op/s 2024-06-02T15:50:00.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:00 smithi090 ceph-mon[31595]: overall HEALTH_OK 2024-06-02T15:50:00.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:00 smithi120 ceph-mon[36451]: overall HEALTH_OK 2024-06-02T15:50:00.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:00 smithi135 ceph-mon[36839]: overall HEALTH_OK 2024-06-02T15:50:01.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:01 smithi090 ceph-mon[31595]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.36k op/s 2024-06-02T15:50:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:01 smithi120 ceph-mon[36451]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.36k op/s 2024-06-02T15:50:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:01 smithi135 ceph-mon[36839]: pgmap v681: 129 pgs: 129 active+clean; 51 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 1.36k op/s 2024-06-02T15:50:02.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:02 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:50:02.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:02 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:50:02.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:02 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:50:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:03 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:50:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:03 smithi120 ceph-mon[36451]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.34k op/s 2024-06-02T15:50:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:03 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:50:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:03 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:50:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:03 smithi135 ceph-mon[36839]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.34k op/s 2024-06-02T15:50:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:03 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:50:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:03 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:50:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:03 smithi090 ceph-mon[31595]: pgmap v682: 129 pgs: 129 active+clean; 55 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 1.34k op/s 2024-06-02T15:50:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:03 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:50:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:05 smithi120 ceph-mon[36451]: pgmap v683: 129 pgs: 129 active+clean; 55 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 1.31k op/s 2024-06-02T15:50:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:05 smithi135 ceph-mon[36839]: pgmap v683: 129 pgs: 129 active+clean; 55 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 1.31k op/s 2024-06-02T15:50:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:05 smithi090 ceph-mon[31595]: pgmap v683: 129 pgs: 129 active+clean; 55 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 1.31k op/s 2024-06-02T15:50:05.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:50:06.339 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:07 smithi120 ceph-mon[36451]: pgmap v684: 129 pgs: 129 active+clean; 65 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.18k op/s 2024-06-02T15:50:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:07 smithi135 ceph-mon[36839]: pgmap v684: 129 pgs: 129 active+clean; 65 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.18k op/s 2024-06-02T15:50:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:07 smithi090 ceph-mon[31595]: pgmap v684: 129 pgs: 129 active+clean; 65 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.18k op/s 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 -- 172.21.15.90:0/2849866913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 msgr2=0x7f861010a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 --2- 172.21.15.90:0/2849866913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f861010a740 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009900 tx=0x7f85fc02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 -- 172.21.15.90:0/2849866913 shutdown_connections 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 --2- 172.21.15.90:0/2849866913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f8610111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 --2- 172.21.15.90:0/2849866913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f861010a740 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 --2- 172.21.15.90:0/2849866913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 0x7f86101094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 -- 172.21.15.90:0/2849866913 >> 172.21.15.90:0/2849866913 conn(0x7f8610078070 msgr2=0x7f861007a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 -- 172.21.15.90:0/2849866913 shutdown_connections 2024-06-02T15:50:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.181+0000 7f8617800640 1 -- 172.21.15.90:0/2849866913 wait complete. 2024-06-02T15:50:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.182+0000 7f8617800640 1 Processor -- start 2024-06-02T15:50:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.182+0000 7f8617800640 1 -- start start 2024-06-02T15:50:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 0x7f8610071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f8610071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8610114c60 con 0x7f861010ac80 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8615800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8614e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 0x7f8610071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8615800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43160/0 (socket says 172.21.15.90:43160) 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8615800640 1 -- 172.21.15.90:0/1023700732 learned_addr learned my addr 172.21.15.90:0/1023700732 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8610114ae0 con 0x7f861010a2c0 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.183+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8610114de0 con 0x7f86101090d0 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f860fe00640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f8610071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 -- 172.21.15.90:0/1023700732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 msgr2=0x7f8610071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 0x7f8610071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 -- 172.21.15.90:0/1023700732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 msgr2=0x7f8610071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f8610071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 -- 172.21.15.90:0/1023700732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101af870 con 0x7f861010ac80 2024-06-02T15:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f860fe00640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f8610071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f8615800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f860800cad0 tx=0x7f860800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f860d600640 1 -- 172.21.15.90:0/1023700732 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800ae20 con 0x7f861010ac80 2024-06-02T15:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f860d600640 1 -- 172.21.15.90:0/1023700732 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8608005020 con 0x7f861010ac80 2024-06-02T15:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.184+0000 7f860d600640 1 -- 172.21.15.90:0/1023700732 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860800f6b0 con 0x7f861010ac80 2024-06-02T15:50:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.185+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86101afa40 con 0x7f861010ac80 2024-06-02T15:50:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.185+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101afe50 con 0x7f861010ac80 2024-06-02T15:50:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.191+0000 7f860d600640 1 -- 172.21.15.90:0/1023700732 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86080051d0 con 0x7f861010ac80 2024-06-02T15:50:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.191+0000 7f861a607740 1 Processor -- start 2024-06-02T15:50:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.192+0000 7f861a607740 1 -- start start 2024-06-02T15:50:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.192+0000 7f860d600640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 0x7f85f80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.192+0000 7f861a607740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 0x564425470f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f860d600640 1 -- 172.21.15.90:0/1023700732 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f86080e2930 con 0x7f861010ac80 2024-06-02T15:50:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f861a607740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x564425476020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f860fe00640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 0x7f85f80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f861a607740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x564425478b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f861a607740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5644253e0b90 con 0x564425471520 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f861a607740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5644253d0730 con 0x564425470b30 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f861a607740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644253d4770 con 0x564425476690 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f8614e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 0x564425470f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f8614e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 0x564425470f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:41706/0 (socket says 172.21.15.90:41706) 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.193+0000 7f8614e00640 1 -- 172.21.15.90:0/1255283293 learned_addr learned my addr 172.21.15.90:0/1255283293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f860fe00640 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x564425476020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x564425478b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 -- 172.21.15.90:0/1255283293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 msgr2=0x564425470f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 0x564425470f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 -- 172.21.15.90:0/1255283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 msgr2=0x564425476020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x564425476020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 -- 172.21.15.90:0/1255283293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644253698b0 con 0x564425476690 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.194+0000 7f8615800640 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x564425478b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86080e0020 tx=0x7f86080a9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112e5b6bb02ec2d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.195+0000 7f860fe00640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 0x7f85f80b4e60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85fc02a910 tx=0x7f85fc0044d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.197+0000 7f8602000640 1 -- 172.21.15.90:0/1255283293 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080b22f0 con 0x564425476690 2024-06-02T15:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.197+0000 7f8602000640 1 -- 172.21.15.90:0/1255283293 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86080adef0 con 0x564425476690 2024-06-02T15:50:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.197+0000 7f8602000640 1 -- 172.21.15.90:0/1255283293 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080a54b0 con 0x564425476690 2024-06-02T15:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.198+0000 7f861a607740 1 -- 172.21.15.90:0/1255283293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 msgr2=0x564425478b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x564425478b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86080e0020 tx=0x7f86080a9790 comp rx=0 tx=0).stop 2024-06-02T15:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 -- 172.21.15.90:0/1255283293 shutdown_connections 2024-06-02T15:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x564425478b20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x564425476020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 --2- 172.21.15.90:0/1255283293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425470b30 0x564425470f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 -- 172.21.15.90:0/1255283293 >> 172.21.15.90:0/1255283293 conn(0x56442546c1a0 msgr2=0x56442546c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 -- 172.21.15.90:0/1255283293 shutdown_connections 2024-06-02T15:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.199+0000 7f861a607740 1 -- 172.21.15.90:0/1255283293 wait complete. 2024-06-02T15:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.200+0000 7f861a607740 1 -- start start 2024-06-02T15:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.200+0000 7f861a607740 1 Processor -- start 2024-06-02T15:50:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5644250b6790 con 0x564425471520 2024-06-02T15:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x564425378bc0 con 0x564425476690 2024-06-02T15:50:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.201+0000 7f861a607740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644254795e0 con 0x564425470b30 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.202+0000 7f8615800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.203+0000 7f8615800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:41714/0 (socket says 172.21.15.90:41714) 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.203+0000 7f8615800640 1 -- 172.21.15.90:0/3311627359 learned_addr learned my addr 172.21.15.90:0/3311627359 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.202+0000 7f8614e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.202+0000 7f861a607740 10 client.0 _ll_register_callbacks cb 0x564425356400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.202+0000 7f860fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 -- 172.21.15.90:0/3311627359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 msgr2=0x56442547a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 -- 172.21.15.90:0/3311627359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 msgr2=0x56442547ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 -- 172.21.15.90:0/3311627359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644250e5210 con 0x564425476690 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.204+0000 7f8615800640 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86080a6db0 tx=0x7f86080b2430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5c73e6960a1d2a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.205+0000 7f85f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.207+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080c2070 con 0x564425476690 2024-06-02T15:50:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.207+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f86080a8fb0 con 0x564425476690 2024-06-02T15:50:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.207+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86080adef0 con 0x564425476690 2024-06-02T15:50:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.208+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86080a54b0 con 0x564425476690 2024-06-02T15:50:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.208+0000 7f861a607740 10 client.25934 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.208+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5644251b1620 con 0x564425476690 2024-06-02T15:50:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.209+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86080adef0 con 0x564425476690 2024-06-02T15:50:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f86080af080 con 0x564425476690 2024-06-02T15:50:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 10 client.25934 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 20 client.25934 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x5644254883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 4 client.25934 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.210+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644253e2ae0 con 0x564425485fd0 2024-06-02T15:50:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.211+0000 7f8614e00640 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x5644254883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.212+0000 7f8614e00640 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x5644254883c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30bb52f3f1998bd1 server_cookie=1f21f091ddeaa1ca in_seq=0 out_seq=0 2024-06-02T15:50:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.212+0000 7f85f6a00640 10 client.25934 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.214+0000 7f85f6a00640 1 -- 172.21.15.90:0/3311627359 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5644253e2ae0 con 0x564425485fd0 2024-06-02T15:50:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.214+0000 7f85f6a00640 10 client.25934 handle_command_reply: tid=0 2024-06-02T15:50:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.214+0000 7f861a607740 1 client.25934 shutdown 2024-06-02T15:50:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 msgr2=0x564425481eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86080a6db0 tx=0x7f86080b2430 comp rx=0 tx=0).stop 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 shutdown_connections 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x5644254883c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f85f6a00640 0 client.25934 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425481eb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 --2- 172.21.15.90:0/3311627359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 >> 172.21.15.90:0/3311627359 conn(0x56442546c1a0 msgr2=0x56442546b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 shutdown_connections 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.215+0000 7f861a607740 1 -- 172.21.15.90:0/3311627359 wait complete. 2024-06-02T15:50:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.216+0000 7f861a607740 20 client.25934 trim_cache size 0 max 16384 2024-06-02T15:50:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.235+0000 7f861a607740 1 Processor -- start 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.235+0000 7f861a607740 1 -- start start 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425470b30 0x56442546aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 0x564425481e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86101bcee0 con 0x564425470b30 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ba750 con 0x564425471520 2024-06-02T15:50:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.236+0000 7f861a607740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b85a0 con 0x564425476690 2024-06-02T15:50:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f860fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 0x564425481e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f8615800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f8615800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60266/0 (socket says 172.21.15.90:60266) 2024-06-02T15:50:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f8615800640 1 -- 172.21.15.90:0/18484377 learned_addr learned my addr 172.21.15.90:0/18484377 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f8614e00640 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425470b30 0x56442546aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f860fe00640 1 -- 172.21.15.90:0/18484377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 msgr2=0x56442546afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f860fe00640 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.237+0000 7f860fe00640 1 -- 172.21.15.90:0/18484377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425470b30 msgr2=0x56442546aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.238+0000 7f860fe00640 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425470b30 0x56442546aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.238+0000 7f860fe00640 1 -- 172.21.15.90:0/18484377 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442546bfd0 con 0x564425471520 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.238+0000 7f8615800640 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.238+0000 7f860fe00640 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 0x564425481e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564425471f90 tx=0x7f85fc042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e11db859480d04c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.239+0000 7f8601600640 1 -- 172.21.15.90:0/18484377 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc04e070 con 0x564425471520 2024-06-02T15:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.239+0000 7f8601600640 1 -- 172.21.15.90:0/18484377 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85fc005590 con 0x564425471520 2024-06-02T15:50:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.239+0000 7f8601600640 1 -- 172.21.15.90:0/18484377 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc035050 con 0x564425471520 2024-06-02T15:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 -- 172.21.15.90:0/18484377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 msgr2=0x564425481e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 0x564425481e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564425471f90 tx=0x7f85fc042000 comp rx=0 tx=0).stop 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 -- 172.21.15.90:0/18484377 shutdown_connections 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425476690 0x56442546afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425471520 0x564425481e70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 --2- 172.21.15.90:0/18484377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425470b30 0x56442546aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.240+0000 7f861a607740 1 -- 172.21.15.90:0/18484377 >> 172.21.15.90:0/18484377 conn(0x56442546c1a0 msgr2=0x564425471fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.241+0000 7f861a607740 1 -- 172.21.15.90:0/18484377 shutdown_connections 2024-06-02T15:50:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.241+0000 7f861a607740 1 -- 172.21.15.90:0/18484377 wait complete. 2024-06-02T15:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.241+0000 7f861a607740 1 -- start start 2024-06-02T15:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 Processor -- start 2024-06-02T15:50:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425480e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b8c10 con 0x564425471520 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ba080 con 0x564425476690 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b92e0 con 0x564425470b30 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f861a607740 10 client.0 _ll_register_callbacks cb 0x7f860400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.242+0000 7f8614e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8614e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60268/0 (socket says 172.21.15.90:60268) 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8614e00640 1 -- 172.21.15.90:0/4043827982 learned_addr learned my addr 172.21.15.90:0/4043827982 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425480e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f860fe00640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 -- 172.21.15.90:0/4043827982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 msgr2=0x56442547d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 -- 172.21.15.90:0/4043827982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 msgr2=0x56442547de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.243+0000 7f8615800640 1 -- 172.21.15.90:0/4043827982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644254700e0 con 0x564425476690 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.244+0000 7f8615800640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425480e00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f86080058d0 tx=0x7f860800a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0e3effdc5bf5e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.244+0000 7f85f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.244+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080c2040 con 0x564425476690 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.244+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f86080a8fb0 con 0x564425476690 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.244+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85fc005590 con 0x564425476690 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.245+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86080a5a00 con 0x564425476690 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.245+0000 7f861a607740 10 client.25940 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.245+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564425501c20 con 0x564425476690 2024-06-02T15:50:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.245+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85fc005590 con 0x564425476690 2024-06-02T15:50:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.245+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f8608004c00 con 0x564425476690 2024-06-02T15:50:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 10 client.25940 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 20 client.25940 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x564425504300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 4 client.25940 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644253dc2c0 con 0x564425485fd0 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.246+0000 7f8614e00640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x564425504300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.247+0000 7f8614e00640 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x564425504300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9404fabe55c259f1 server_cookie=31a6fb2b4ccc934e in_seq=0 out_seq=0 2024-06-02T15:50:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:08.247+0000 7f85f6a00640 10 client.25940 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:08.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:08 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:50:08.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:08 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:50:08.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:08 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:50:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:09 smithi120 ceph-mon[36451]: pgmap v685: 129 pgs: 129 active+clean; 69 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 522 op/s 2024-06-02T15:50:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:09 smithi135 ceph-mon[36839]: pgmap v685: 129 pgs: 129 active+clean; 69 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 522 op/s 2024-06-02T15:50:09.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:09 smithi090 ceph-mon[31595]: pgmap v685: 129 pgs: 129 active+clean; 69 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 522 op/s 2024-06-02T15:50:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.670+0000 7f85f6a00640 1 -- 172.21.15.90:0/4043827982 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5644253dc2c0 con 0x564425485fd0 2024-06-02T15:50:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.670+0000 7f85f6a00640 10 client.25940 handle_command_reply: tid=0 2024-06-02T15:50:10.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.670+0000 7f861a607740 1 client.25940 shutdown 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 msgr2=0x564425480e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425480e00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f86080058d0 tx=0x7f860800a890 comp rx=0 tx=0).stop 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 shutdown_connections 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564425485fd0 0x564425504300 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x564425476690 0x564425480e00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564425471520 0x56442547de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 --2- 172.21.15.90:0/4043827982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564425470b30 0x56442547d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.671+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 >> 172.21.15.90:0/4043827982 conn(0x56442546c1a0 msgr2=0x56442546ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.672+0000 7f85f6a00640 0 client.25940 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.672+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 shutdown_connections 2024-06-02T15:50:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.672+0000 7f861a607740 1 -- 172.21.15.90:0/4043827982 wait complete. 2024-06-02T15:50:10.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.672+0000 7f861a607740 20 client.25940 trim_cache size 0 max 16384 2024-06-02T15:50:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:50:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:50:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:50:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.675+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 msgr2=0x7f85f80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.675+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 0x7f85f80b4e60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85fc02a910 tx=0x7f85fc0044d0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.675+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 msgr2=0x7f86101af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.675+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f860800cad0 tx=0x7f860800cf90 comp rx=0 tx=0).stop 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 shutdown_connections 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85f80b29a0 0x7f85f80b4e60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f861010ac80 0x7f86101af330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f861010a2c0 0x7f8610071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 --2- 172.21.15.90:0/1023700732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101090d0 0x7f8610071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 >> 172.21.15.90:0/1023700732 conn(0x7f8610078070 msgr2=0x7f861010d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.676+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 shutdown_connections 2024-06-02T15:50:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:10.677+0000 7f8617800640 1 -- 172.21.15.90:0/1023700732 wait complete. 2024-06-02T15:50:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:11 smithi120 ceph-mon[36451]: pgmap v686: 129 pgs: 129 active+clean; 69 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-06-02T15:50:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:11 smithi135 ceph-mon[36839]: pgmap v686: 129 pgs: 129 active+clean; 69 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-06-02T15:50:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:11 smithi090 ceph-mon[31595]: pgmap v686: 129 pgs: 129 active+clean; 69 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 398 op/s 2024-06-02T15:50:12.266 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[66356]: *** Caught signal (Segmentation fault) ** 2024-06-02T15:50:12.323 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[66356]: in thread 7fad0236a700 thread_name:MR_Finisher 2024-06-02T15:50:12.323 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:12 smithi120 podman[69951]: 2024-06-02 15:50:12.009610938 +0000 UTC m=+0.019970199 container died 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:50:13.191 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:12 smithi120 podman[69951]: 2024-06-02 15:50:12.941338391 +0000 UTC m=+0.951697640 container remove 48371ec6b816e01aea297d8252ccb742d927e3ff7c62826cf681e19fffcd03d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-02T15:50:13.191 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:12 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-06-02T15:50:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:13 smithi120 ceph-mon[36451]: pgmap v687: 129 pgs: 129 active+clean; 75 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 927 op/s 2024-06-02T15:50:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:50:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:50:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:50:13.557 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:13 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-06-02T15:50:13.558 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:13 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service: Consumed 6.607s CPU time. 2024-06-02T15:50:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:13 smithi135 ceph-mon[36839]: pgmap v687: 129 pgs: 129 active+clean; 75 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 927 op/s 2024-06-02T15:50:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:50:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:50:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:50:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:13 smithi090 ceph-mon[31595]: pgmap v687: 129 pgs: 129 active+clean; 75 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 927 op/s 2024-06-02T15:50:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:50:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:50:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:50:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:15 smithi120 ceph-mon[36451]: pgmap v688: 129 pgs: 129 active+clean; 75 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 1.06k op/s 2024-06-02T15:50:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:15 smithi135 ceph-mon[36839]: pgmap v688: 129 pgs: 129 active+clean; 75 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 1.06k op/s 2024-06-02T15:50:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:15 smithi090 ceph-mon[31595]: pgmap v688: 129 pgs: 129 active+clean; 75 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 1.06k op/s 2024-06-02T15:50:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:17 smithi120 ceph-mon[36451]: pgmap v689: 129 pgs: 129 active+clean; 73 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 1.37k op/s 2024-06-02T15:50:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:17 smithi135 ceph-mon[36839]: pgmap v689: 129 pgs: 129 active+clean; 73 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 1.37k op/s 2024-06-02T15:50:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:17 smithi090 ceph-mon[31595]: pgmap v689: 129 pgs: 129 active+clean; 73 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 1.37k op/s 2024-06-02T15:50:19.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:19 smithi120 ceph-mon[36451]: pgmap v690: 129 pgs: 129 active+clean; 75 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 1.32k op/s 2024-06-02T15:50:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:19 smithi135 ceph-mon[36839]: pgmap v690: 129 pgs: 129 active+clean; 75 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 1.32k op/s 2024-06-02T15:50:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:19 smithi090 ceph-mon[31595]: pgmap v690: 129 pgs: 129 active+clean; 75 MiB data, 842 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 1.32k op/s 2024-06-02T15:50:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:21 smithi120 ceph-mon[36451]: pgmap v691: 129 pgs: 129 active+clean; 80 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.30k op/s 2024-06-02T15:50:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:21 smithi135 ceph-mon[36839]: pgmap v691: 129 pgs: 129 active+clean; 80 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.30k op/s 2024-06-02T15:50:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:21 smithi090 ceph-mon[31595]: pgmap v691: 129 pgs: 129 active+clean; 80 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 1.30k op/s 2024-06-02T15:50:22.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:22 smithi120 ceph-mon[36451]: pgmap v692: 129 pgs: 129 active+clean; 89 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 1.07k op/s 2024-06-02T15:50:22.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:22 smithi135 ceph-mon[36839]: pgmap v692: 129 pgs: 129 active+clean; 89 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 1.07k op/s 2024-06-02T15:50:22.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:22 smithi090 ceph-mon[31595]: pgmap v692: 129 pgs: 129 active+clean; 89 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 1.07k op/s 2024-06-02T15:50:23.807 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:23 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-06-02T15:50:23.807 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:23 smithi120 systemd[1]: Stopped Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:50:23.807 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:23 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service: Consumed 6.607s CPU time. 2024-06-02T15:50:23.807 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:23 smithi120 systemd[1]: Starting Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:50:23.807 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:23 smithi120 podman[70085]: 2024-06-02 15:50:23.446514686 +0000 UTC m=+0.018947996 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:50:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.865+0000 7f2b6fe00640 1 -- 172.21.15.90:0/1384918005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b7010eeb0 msgr2=0x7f2b701112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.865+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/1384918005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b7010eeb0 0x7f2b701112a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009890 tx=0x7f2b6002a370 comp rx=0 tx=0).stop 2024-06-02T15:50:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 -- 172.21.15.90:0/1384918005 shutdown_connections 2024-06-02T15:50:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/1384918005 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b7010eeb0 0x7f2b701112a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/1384918005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b70075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/1384918005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b70076a90 0x7f2b70074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 -- 172.21.15.90:0/1384918005 >> 172.21.15.90:0/1384918005 conn(0x7f2b700fde50 msgr2=0x7f2b70100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 -- 172.21.15.90:0/1384918005 shutdown_connections 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.866+0000 7f2b6fe00640 1 -- 172.21.15.90:0/1384918005 wait complete. 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 Processor -- start 2024-06-02T15:50:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 -- start start 2024-06-02T15:50:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 0x7f2b7019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 0x7f2b701a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114a00 con 0x7f2b700754b0 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114880 con 0x7f2b70076a90 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.868+0000 7f2b6fe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70114b80 con 0x7f2b7010eeb0 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59534/0 (socket says 172.21.15.90:59534) 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6ea00640 1 -- 172.21.15.90:0/3920605956 learned_addr learned my addr 172.21.15.90:0/3920605956 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6e000640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 0x7f2b7019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6f400640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 0x7f2b701a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.869+0000 7f2b6ea00640 1 -- 172.21.15.90:0/3920605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 msgr2=0x7f2b701a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 0x7f2b701a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6ea00640 1 -- 172.21.15.90:0/3920605956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 msgr2=0x7f2b7019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 0x7f2b7019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6ea00640 1 -- 172.21.15.90:0/3920605956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b701a6d30 con 0x7f2b700754b0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c420 tx=0x7f2b6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6f400640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 0x7f2b701a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.870+0000 7f2b6e000640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 0x7f2b7019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.871+0000 7f2b67400640 1 -- 172.21.15.90:0/3920605956 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004b20 con 0x7f2b700754b0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.871+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701a6fc0 con 0x7f2b700754b0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.871+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b701a7480 con 0x7f2b700754b0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.871+0000 7f2b67400640 1 -- 172.21.15.90:0/3920605956 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b68003e20 con 0x7f2b700754b0 2024-06-02T15:50:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.871+0000 7f2b67400640 1 -- 172.21.15.90:0/3920605956 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800b630 con 0x7f2b700754b0 2024-06-02T15:50:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.875+0000 7f2b787ef740 1 Processor -- start 2024-06-02T15:50:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 -- start start 2024-06-02T15:50:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f26e0 0x55fee24f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 0x55fee24f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fee2461b90 con 0x55fee24f7850 2024-06-02T15:50:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.876+0000 7f2b787ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fee2451730 con 0x55fee24f26e0 2024-06-02T15:50:23.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.877+0000 7f2b787ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fee2455770 con 0x55fee24f1cf0 2024-06-02T15:50:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.877+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.877+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55000/0 (socket says 172.21.15.90:55000) 2024-06-02T15:50:23.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.877+0000 7f2b6f400640 1 -- 172.21.15.90:0/2517059667 learned_addr learned my addr 172.21.15.90:0/2517059667 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:23.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.878+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 0x55fee24f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.879+0000 7f2b6e000640 1 -- 172.21.15.90:0/2517059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 msgr2=0x55fee24f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.879+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.879+0000 7f2b6e000640 1 -- 172.21.15.90:0/2517059667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f26e0 msgr2=0x55fee24f71e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.879+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f26e0 0x55fee24f71e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.879+0000 7f2b6e000640 1 -- 172.21.15.90:0/2517059667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee23ea8b0 con 0x55fee24f7850 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.880+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 0x55fee24f9ce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005d00 tx=0x7f2b5c010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edef70b84deeeca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:23.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.880+0000 7f2b5be00640 1 -- 172.21.15.90:0/2517059667 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011760 con 0x55fee24f7850 2024-06-02T15:50:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.880+0000 7f2b5be00640 1 -- 172.21.15.90:0/2517059667 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c011db0 con 0x55fee24f7850 2024-06-02T15:50:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.880+0000 7f2b5be00640 1 -- 172.21.15.90:0/2517059667 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01ae30 con 0x55fee24f7850 2024-06-02T15:50:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.881+0000 7f2b67400640 1 -- 172.21.15.90:0/3920605956 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2b6801d020 con 0x7f2b700754b0 2024-06-02T15:50:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.881+0000 7f2b787ef740 1 -- 172.21.15.90:0/2517059667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 msgr2=0x55fee24f9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.881+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 0x55fee24f9ce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005d00 tx=0x7f2b5c010930 comp rx=0 tx=0).stop 2024-06-02T15:50:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 -- 172.21.15.90:0/2517059667 shutdown_connections 2024-06-02T15:50:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f7850 0x55fee24f9ce0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f26e0 0x55fee24f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2517059667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 -- 172.21.15.90:0/2517059667 >> 172.21.15.90:0/2517059667 conn(0x55fee24ed360 msgr2=0x55fee24ed760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.882+0000 7f2b787ef740 1 -- 172.21.15.90:0/2517059667 shutdown_connections 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.883+0000 7f2b787ef740 1 -- 172.21.15.90:0/2517059667 wait complete. 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.883+0000 7f2b787ef740 1 -- start start 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.883+0000 7f2b67400640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 0x7f2b540b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b787ef740 1 Processor -- start 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b67400640 1 -- 172.21.15.90:0/3920605956 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b680e25d0 con 0x7f2b700754b0 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b6f400640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 0x7f2b540b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f1cf0 0x55fee250cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee25122f0 0x55fee25146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.884+0000 7f2b787ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fee2137790 con 0x55fee25122f0 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b787ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fee23f9bc0 con 0x55fee24f1cf0 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b787ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fee250b810 con 0x55fee24f26e0 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b787ef740 10 client.0 _ll_register_callbacks cb 0x55fee23d7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55016/0 (socket says 172.21.15.90:55016) 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 -- 172.21.15.90:0/716353760 learned_addr learned my addr 172.21.15.90:0/716353760 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f1cf0 0x55fee250cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 -- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f1cf0 msgr2=0x55fee250cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f1cf0 0x55fee250cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 -- 172.21.15.90:0/716353760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee25122f0 msgr2=0x55fee25146e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee25122f0 0x55fee25146e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.885+0000 7f2b6e000640 1 -- 172.21.15.90:0/716353760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee2166210 con 0x55fee24f26e0 2024-06-02T15:50:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b6f400640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 0x7f2b540b4bb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002a960 tx=0x7f2b60005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b6e000640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000c00 tx=0x7f2b5c01ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0c1a2ea0e76820 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c029020 con 0x55fee24f26e0 2024-06-02T15:50:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b5c004020 con 0x55fee24f26e0 2024-06-02T15:50:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.886+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c011db0 con 0x55fee24f26e0 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c01a790 con 0x55fee24f26e0 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b787ef740 10 client.25906 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fee2232620 con 0x55fee24f26e0 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c011db0 con 0x55fee24f26e0 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2b5c024070 con 0x55fee24f26e0 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b787ef740 10 client.25906 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.887+0000 7f2b787ef740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.888+0000 7f2b787ef740 20 client.25906 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.888+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2518aa0 0x55fee251ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.888+0000 7f2b787ef740 4 client.25906 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.888+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fee2463ae0 con 0x55fee2518aa0 2024-06-02T15:50:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.975+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2518aa0 0x55fee251ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.976+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2518aa0 0x55fee251ae90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895e59917d1db102 server_cookie=69a0caded04c3468 in_seq=0 out_seq=0 2024-06-02T15:50:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.976+0000 7f2b58c00640 10 client.25906 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b58c00640 1 -- 172.21.15.90:0/716353760 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fee2463ae0 con 0x55fee2518aa0 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b58c00640 10 client.25906 handle_command_reply: tid=0 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 client.25906 shutdown 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 msgr2=0x55fee250d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c000c00 tx=0x7f2b5c01ade0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 shutdown_connections 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2518aa0 0x55fee251ae90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee25122f0 0x55fee25146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f26e0 0x55fee250d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.979+0000 7f2b787ef740 1 --2- 172.21.15.90:0/716353760 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee24f1cf0 0x55fee250cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.980+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 >> 172.21.15.90:0/716353760 conn(0x55fee24ed360 msgr2=0x55fee24ec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.980+0000 7f2b58c00640 0 client.25906 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.980+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 shutdown_connections 2024-06-02T15:50:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.980+0000 7f2b787ef740 1 -- 172.21.15.90:0/716353760 wait complete. 2024-06-02T15:50:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:23.980+0000 7f2b787ef740 20 client.25906 trim_cache size 0 max 16384 2024-06-02T15:50:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.003+0000 7f2b787ef740 1 Processor -- start 2024-06-02T15:50:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 -- start start 2024-06-02T15:50:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee24f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50012570 con 0x55fee24f26e0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000fde0 con 0x55fee25122f0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b787ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000dc30 con 0x55fee24f1cf0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59544/0 (socket says 172.21.15.90:59544) 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b6e000640 1 -- 172.21.15.90:0/2880175550 learned_addr learned my addr 172.21.15.90:0/2880175550 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b6f400640 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee24f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.004+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 -- 172.21.15.90:0/2880175550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 msgr2=0x55fee24f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 -- 172.21.15.90:0/2880175550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 msgr2=0x55fee24f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee24f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 -- 172.21.15.90:0/2880175550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee24f1290 con 0x55fee24f26e0 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b6e000640 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c003900 tx=0x7f2b5c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a136afb795bfb99 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.005+0000 7f2b5b400640 1 -- 172.21.15.90:0/2880175550 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c039070 con 0x55fee24f26e0 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.006+0000 7f2b5b400640 1 -- 172.21.15.90:0/2880175550 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c003b80 con 0x55fee24f26e0 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.006+0000 7f2b5b400640 1 -- 172.21.15.90:0/2880175550 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c011560 con 0x55fee24f26e0 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.006+0000 7f2b787ef740 1 -- 172.21.15.90:0/2880175550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 msgr2=0x55fee24f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.006+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c003900 tx=0x7f2b5c001e50 comp rx=0 tx=0).stop 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 -- 172.21.15.90:0/2880175550 shutdown_connections 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee24f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f26e0 0x55fee24f3180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 --2- 172.21.15.90:0/2880175550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee24f1cf0 0x55fee24f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 -- 172.21.15.90:0/2880175550 >> 172.21.15.90:0/2880175550 conn(0x55fee24ed360 msgr2=0x55fee24f00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 -- 172.21.15.90:0/2880175550 shutdown_connections 2024-06-02T15:50:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 -- 172.21.15.90:0/2880175550 wait complete. 2024-06-02T15:50:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.007+0000 7f2b787ef740 1 -- start start 2024-06-02T15:50:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.008+0000 7f2b787ef740 1 Processor -- start 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f1cf0 0x55fee252a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee252bae0 0x55fee252bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000e2a0 con 0x55fee24f1cf0 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000f710 con 0x55fee25122f0 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000e970 con 0x55fee252bae0 2024-06-02T15:50:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b787ef740 10 client.0 _ll_register_callbacks cb 0x7f2b60035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b6ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f1cf0 0x55fee252a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b6e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34066/0 (socket says 172.21.15.90:34066) 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.009+0000 7f2b6e000640 1 -- 172.21.15.90:0/4037170080 learned_addr learned my addr 172.21.15.90:0/4037170080 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 -- 172.21.15.90:0/4037170080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee252bae0 msgr2=0x55fee252bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6f400640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee252bae0 0x55fee252bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee252bae0 0x55fee252bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 -- 172.21.15.90:0/4037170080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f1cf0 msgr2=0x55fee252a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f1cf0 0x55fee252a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 -- 172.21.15.90:0/4037170080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee252c4a0 con 0x55fee25122f0 2024-06-02T15:50:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.010+0000 7f2b6e000640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c010970 tx=0x7f2b5c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe3d1be0c30e7ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b58c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c039070 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2b5c01a590 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c003b80 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c01a790 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b787ef740 10 client.25955 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fee2583a70 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.011+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5c003b80 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2b5c045070 con 0x55fee25122f0 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 10 client.25955 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 20 client.25955 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2584380 0x55fee25867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 4 client.25955 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fee245d2c0 con 0x55fee2584380 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.012+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2584380 0x55fee25867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.013+0000 7f2b6ea00640 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2584380 0x55fee25867a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b206b796e92eb7 server_cookie=48424f6551ccf2e9 in_seq=0 out_seq=0 2024-06-02T15:50:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:24.013+0000 7f2b58c00640 10 client.25955 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.608+0000 7fb13ba00700 1 -- 172.21.15.90:0/1056139969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 msgr2=0x7fb134075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.608+0000 7fb13ba00700 1 --2- 172.21.15.90:0/1056139969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb134075b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009990 tx=0x7fb12c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.609+0000 7fb13ba00700 1 -- 172.21.15.90:0/1056139969 shutdown_connections 2024-06-02T15:50:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.609+0000 7fb13ba00700 1 --2- 172.21.15.90:0/1056139969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134114710 0x7fb134116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.609+0000 7fb13ba00700 1 --2- 172.21.15.90:0/1056139969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb134075b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.609+0000 7fb13ba00700 1 --2- 172.21.15.90:0/1056139969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb134076090 0x7fb1341141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.609+0000 7fb13ba00700 1 -- 172.21.15.90:0/1056139969 >> 172.21.15.90:0/1056139969 conn(0x7fb13407a6a0 msgr2=0x7fb13407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.610+0000 7fb13ba00700 1 -- 172.21.15.90:0/1056139969 shutdown_connections 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.610+0000 7fb13ba00700 1 -- 172.21.15.90:0/1056139969 wait complete. 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.611+0000 7fb13ba00700 1 Processor -- start 2024-06-02T15:50:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.612+0000 7fb13ba00700 1 -- start start 2024-06-02T15:50:24.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.614+0000 7fb13ba00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb1341a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb13ba00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 0x7fb1341a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb13ba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb134114710 0x7fb1341a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb13ba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb134119d50 con 0x7fb134114710 2024-06-02T15:50:24.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb13ba00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb134119bd0 con 0x7fb134075770 2024-06-02T15:50:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb13ba00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb134119ed0 con 0x7fb134076090 2024-06-02T15:50:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.616+0000 7fb139000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb1341a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.617+0000 7fb139000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb1341a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34080/0 (socket says 172.21.15.90:34080) 2024-06-02T15:50:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.617+0000 7fb139000700 1 -- 172.21.15.90:0/2154450486 learned_addr learned my addr 172.21.15.90:0/2154450486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.617+0000 7fb133e00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 0x7fb1341a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.618+0000 7fb133e00700 1 -- 172.21.15.90:0/2154450486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 msgr2=0x7fb1341a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.618+0000 7fb133e00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb1341a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.618+0000 7fb133e00700 1 -- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb134114710 msgr2=0x7fb1341a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.618+0000 7fb133e00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb134114710 0x7fb1341a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.618+0000 7fb133e00700 1 -- 172.21.15.90:0/2154450486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1341a5270 con 0x7fb134076090 2024-06-02T15:50:24.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.619+0000 7fb133e00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 0x7fb1341a07a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.619+0000 7fb131600700 1 -- 172.21.15.90:0/2154450486 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c017070 con 0x7fb134076090 2024-06-02T15:50:24.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.620+0000 7fb131600700 1 -- 172.21.15.90:0/2154450486 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb11c009de0 con 0x7fb134076090 2024-06-02T15:50:24.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.620+0000 7fb131600700 1 -- 172.21.15.90:0/2154450486 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c010e20 con 0x7fb134076090 2024-06-02T15:50:24.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.620+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1341a5560 con 0x7fb134076090 2024-06-02T15:50:24.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.620+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1341acde0 con 0x7fb134076090 2024-06-02T15:50:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.634+0000 7fb146356b80 1 Processor -- start 2024-06-02T15:50:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.635+0000 7fb131600700 1 -- 172.21.15.90:0/2154450486 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb11c01a070 con 0x7fb134076090 2024-06-02T15:50:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.637+0000 7fb131600700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 0x7fb1180b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.637+0000 7fb131600700 1 -- 172.21.15.90:0/2154450486 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb11c0e1f20 con 0x7fb134076090 2024-06-02T15:50:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.638+0000 7fb146356b80 1 -- start start 2024-06-02T15:50:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.638+0000 7fb139a00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 0x7fb1180b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.640+0000 7fb139a00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 0x7fb1180b4e40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb124009710 tx=0x7fb124009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.640+0000 7fb146356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.640+0000 7fb146356b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a96467d0 0x55f7a9640260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb146356b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 0x55f7a9640ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb146356b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7a959ee40 con 0x55f7a96467d0 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb146356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7a958e9e0 con 0x55f7a9645eb0 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb146356b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9592a20 con 0x55f7a9640880 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34092/0 (socket says 172.21.15.90:34092) 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 -- 172.21.15.90:0/1416517590 learned_addr learned my addr 172.21.15.90:0/1416517590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139a00700 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 0x55f7a9640ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 -- 172.21.15.90:0/1416517590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 msgr2=0x55f7a9640ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 0x55f7a9640ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 -- 172.21.15.90:0/1416517590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a96467d0 msgr2=0x55f7a9640260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.641+0000 7fb139000700 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a96467d0 0x55f7a9640260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.642+0000 7fb139000700 1 -- 172.21.15.90:0/1416517590 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a9557420 con 0x55f7a9645eb0 2024-06-02T15:50:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.642+0000 7fb139a00700 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 0x55f7a9640ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:24.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.642+0000 7fb139000700 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005bc0 tx=0x7fb12c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740a1a14b6460858 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.642+0000 7fb12a000700 1 -- 172.21.15.90:0/1416517590 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c003a60 con 0x55f7a9645eb0 2024-06-02T15:50:24.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.643+0000 7fb12a000700 1 -- 172.21.15.90:0/1416517590 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c037070 con 0x55f7a9645eb0 2024-06-02T15:50:24.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.644+0000 7fb12a000700 1 -- 172.21.15.90:0/1416517590 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c005360 con 0x55f7a9645eb0 2024-06-02T15:50:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.644+0000 7fb146356b80 1 -- 172.21.15.90:0/1416517590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 msgr2=0x55f7a9646290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.644+0000 7fb146356b80 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb12c005bc0 tx=0x7fb12c005bf0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 -- 172.21.15.90:0/1416517590 shutdown_connections 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a9640880 0x55f7a9640ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9646290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 --2- 172.21.15.90:0/1416517590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a96467d0 0x55f7a9640260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 -- 172.21.15.90:0/1416517590 >> 172.21.15.90:0/1416517590 conn(0x55f7a956ebe0 msgr2=0x55f7a963b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.645+0000 7fb146356b80 1 -- 172.21.15.90:0/1416517590 shutdown_connections 2024-06-02T15:50:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.646+0000 7fb146356b80 1 -- 172.21.15.90:0/1416517590 wait complete. 2024-06-02T15:50:24.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.647+0000 7fb146356b80 1 -- start start 2024-06-02T15:50:24.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.648+0000 7fb146356b80 1 Processor -- start 2024-06-02T15:50:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.650+0000 7fb146356b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a964c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9649830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb139000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a964c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9649d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7a9494fa0 con 0x55f7a9640880 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7a952f760 con 0x55f7a9645eb0 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f7a964d280 con 0x55f7a96467d0 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb146356b80 10 client.0 _ll_register_callbacks cb 0x55f7a9538c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.652+0000 7fb139000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a964c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59580/0 (socket says 172.21.15.90:59580) 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.653+0000 7fb139000700 1 -- 172.21.15.90:0/1021496423 learned_addr learned my addr 172.21.15.90:0/1021496423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.653+0000 7fb139a00700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9649d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.653+0000 7fb133e00700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9649830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.654+0000 7fb133e00700 1 -- 172.21.15.90:0/1021496423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 msgr2=0x55f7a9649d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.654+0000 7fb133e00700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9649d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.654+0000 7fb133e00700 1 -- 172.21.15.90:0/1021496423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 msgr2=0x55f7a964c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.654+0000 7fb133e00700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a964c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.657+0000 7fb133e00700 1 -- 172.21.15.90:0/1021496423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a94de8d0 con 0x55f7a9645eb0 2024-06-02T15:50:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.659+0000 7fb133e00700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9649830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0df5e0 tx=0x7fb11c00df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc8fa466c2a9ad81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.659+0000 7fb122a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.659+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c021050 con 0x55f7a9645eb0 2024-06-02T15:50:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.659+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb11c0b1d10 con 0x55f7a9645eb0 2024-06-02T15:50:24.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.660+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12c037070 con 0x55f7a9645eb0 2024-06-02T15:50:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.660+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb11c0a9130 con 0x55f7a9645eb0 2024-06-02T15:50:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.660+0000 7fb146356b80 10 client.25967 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.660+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7a955f060 con 0x55f7a9645eb0 2024-06-02T15:50:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.660+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c037070 con 0x55f7a9645eb0 2024-06-02T15:50:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.661+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb11c0a8390 con 0x55f7a9645eb0 2024-06-02T15:50:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.661+0000 7fb146356b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.661+0000 7fb146356b80 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.661+0000 7fb146356b80 20 client.25967 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.661+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a9659ba0 0x55f7a965bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.662+0000 7fb146356b80 4 client.25967 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:24.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.662+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7a95a0d90 con 0x55f7a9659ba0 2024-06-02T15:50:24.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.666+0000 7fb139000700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a9659ba0 0x55f7a965bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.669+0000 7fb139000700 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a9659ba0 0x55f7a965bf90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4927e9c29253381 server_cookie=5d9d0eaaede9250d in_seq=0 out_seq=0 2024-06-02T15:50:24.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.669+0000 7fb122a00700 10 client.25967 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.673+0000 7fb122a00700 1 -- 172.21.15.90:0/1021496423 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7a95a0d90 con 0x55f7a9659ba0 2024-06-02T15:50:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.673+0000 7fb122a00700 10 client.25967 handle_command_reply: tid=0 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.673+0000 7fb146356b80 1 client.25967 shutdown 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 msgr2=0x55f7a9649830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9649830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0df5e0 tx=0x7fb11c00df90 comp rx=0 tx=0).stop 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 shutdown_connections 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a9659ba0 0x55f7a965bf90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9649d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a964c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 --2- 172.21.15.90:0/1021496423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9649830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb122a00700 0 client.25967 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 >> 172.21.15.90:0/1021496423 conn(0x55f7a956ebe0 msgr2=0x55f7a963f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.674+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 shutdown_connections 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.675+0000 7fb146356b80 1 -- 172.21.15.90:0/1021496423 wait complete. 2024-06-02T15:50:24.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.675+0000 7fb146356b80 20 client.25967 trim_cache size 0 max 16384 2024-06-02T15:50:24.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.719+0000 7fb146356b80 1 Processor -- start 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 -- start start 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 0x55f7a9640260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9645eb0 0x55f7a9634680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341afbd0 con 0x55f7a9645eb0 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341ad400 con 0x55f7a9640880 2024-06-02T15:50:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb146356b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb134073950 con 0x55f7a96467d0 2024-06-02T15:50:24.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb139a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb139000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 0x55f7a9640260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.720+0000 7fb139a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55092/0 (socket says 172.21.15.90:55092) 2024-06-02T15:50:24.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139a00700 1 -- 172.21.15.90:0/3195260512 learned_addr learned my addr 172.21.15.90:0/3195260512 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb133e00700 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9645eb0 0x55f7a9634680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139000700 1 -- 172.21.15.90:0/3195260512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 msgr2=0x55f7a9647380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139000700 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139000700 1 -- 172.21.15.90:0/3195260512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9645eb0 msgr2=0x55f7a9634680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139000700 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9645eb0 0x55f7a9634680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.721+0000 7fb139000700 1 -- 172.21.15.90:0/3195260512 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a963bd30 con 0x55f7a9640880 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.723+0000 7fb139a00700 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.724+0000 7fb139000700 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 0x55f7a9640260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00b5c0 tx=0x7fb12c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3190e43378a90787 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.725+0000 7fb129600700 1 -- 172.21.15.90:0/3195260512 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00bad0 con 0x55f7a9640880 2024-06-02T15:50:24.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.725+0000 7fb129600700 1 -- 172.21.15.90:0/3195260512 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c037030 con 0x55f7a9640880 2024-06-02T15:50:24.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.726+0000 7fb129600700 1 -- 172.21.15.90:0/3195260512 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00f8c0 con 0x55f7a9640880 2024-06-02T15:50:24.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.727+0000 7fb146356b80 1 -- 172.21.15.90:0/3195260512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 msgr2=0x55f7a9640260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.727+0000 7fb146356b80 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 0x55f7a9640260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00b5c0 tx=0x7fb12c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 -- 172.21.15.90:0/3195260512 shutdown_connections 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a9647380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9640880 0x55f7a9640260 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 --2- 172.21.15.90:0/3195260512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9645eb0 0x55f7a9634680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 -- 172.21.15.90:0/3195260512 >> 172.21.15.90:0/3195260512 conn(0x55f7a956ebe0 msgr2=0x55f7a9646c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 -- 172.21.15.90:0/3195260512 shutdown_connections 2024-06-02T15:50:24.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.729+0000 7fb146356b80 1 -- 172.21.15.90:0/3195260512 wait complete. 2024-06-02T15:50:24.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.730+0000 7fb146356b80 1 -- start start 2024-06-02T15:50:24.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.730+0000 7fb146356b80 1 Processor -- start 2024-06-02T15:50:24.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.731+0000 7fb146356b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a96527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.731+0000 7fb146356b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb133e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb133e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34114/0 (socket says 172.21.15.90:34114) 2024-06-02T15:50:24.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb146356b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a964f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb133e00700 1 -- 172.21.15.90:0/3863161636 learned_addr learned my addr 172.21.15.90:0/3863161636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb134073fc0 con 0x55f7a9640880 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb134075430 con 0x55f7a9645eb0 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb134074690 con 0x55f7a96467d0 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb139a00700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a964f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.732+0000 7fb146356b80 10 client.0 _ll_register_callbacks cb 0x55f7a9538c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.733+0000 7fb133e00700 1 -- 172.21.15.90:0/3863161636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 msgr2=0x55f7a964f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.733+0000 7fb133e00700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a964f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.733+0000 7fb133e00700 1 -- 172.21.15.90:0/3863161636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 msgr2=0x55f7a96527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.733+0000 7fb133e00700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a96527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.733+0000 7fb133e00700 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a964fdc0 con 0x55f7a9645eb0 2024-06-02T15:50:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.734+0000 7fb122a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.734+0000 7fb133e00700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0ca330 tx=0x7fb11c0a5070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a41bd2bbae6270 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.734+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c021070 con 0x55f7a9645eb0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.734+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb11c0a4aa0 con 0x55f7a9645eb0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.734+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12c037030 con 0x55f7a9645eb0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.735+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb11c0ad4e0 con 0x55f7a9645eb0 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.735+0000 7fb146356b80 10 client.25973 fetch_fsmap learned FSMap version 33 2024-06-02T15:50:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.735+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f7a96c51e0 con 0x55f7a9645eb0 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.735+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c037030 con 0x55f7a9645eb0 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb11c0dfc60 con 0x55f7a9645eb0 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 33 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 20 client.25973 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a96c57e0 0x55f7a96c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 4 client.25973 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:50:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.736+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7a959a570 con 0x55f7a96c57e0 2024-06-02T15:50:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.805+0000 7fb139000700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a96c57e0 0x55f7a96c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:24.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.807+0000 7fb139000700 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a96c57e0 0x55f7a96c7c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542001ba32244c4d server_cookie=4464cc99d9d6fd9f in_seq=0 out_seq=0 2024-06-02T15:50:24.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.807+0000 7fb122a00700 10 client.25973 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.808+0000 7fb122a00700 1 -- 172.21.15.90:0/3863161636 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f7a959a570 con 0x55f7a96c57e0 2024-06-02T15:50:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.808+0000 7fb122a00700 10 client.25973 handle_command_reply: tid=0 2024-06-02T15:50:24.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.808+0000 7fb146356b80 1 client.25973 shutdown 2024-06-02T15:50:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.809+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 msgr2=0x55f7a9652d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.809+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0ca330 tx=0x7fb11c0a5070 comp rx=0 tx=0).stop 2024-06-02T15:50:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.809+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 shutdown_connections 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.809+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7a96c57e0 0x55f7a96c7c00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.810+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7a96467d0 0x55f7a964f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.810+0000 7fb122a00700 0 client.25973 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.810+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7a9640880 0x55f7a96527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.810+0000 7fb146356b80 1 --2- 172.21.15.90:0/3863161636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7a9645eb0 0x55f7a9652d20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.810+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 >> 172.21.15.90:0/3863161636 conn(0x55f7a956ebe0 msgr2=0x55f7a9648120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.811+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 shutdown_connections 2024-06-02T15:50:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.811+0000 7fb146356b80 1 -- 172.21.15.90:0/3863161636 wait complete. 2024-06-02T15:50:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.811+0000 7fb146356b80 20 client.25973 trim_cache size 0 max 16384 2024-06-02T15:50:24.811 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:50:24.811 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:50:24.811 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:50:24.811 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:50:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.813+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 msgr2=0x7fb1180b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.813+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 0x7fb1180b4e40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb124009710 tx=0x7fb124009620 comp rx=0 tx=0).stop 2024-06-02T15:50:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 msgr2=0x7fb1341a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 0x7fb1341a07a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00d950 tx=0x7fb11c00dc60 comp rx=0 tx=0).stop 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 shutdown_connections 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb1180b2980 0x7fb1180b4e40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb134114710 0x7fb1341a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb134075770 0x7fb1341a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 --2- 172.21.15.90:0/2154450486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb134076090 0x7fb1341a07a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 >> 172.21.15.90:0/2154450486 conn(0x7fb13407a6a0 msgr2=0x7fb1341135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.814+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 shutdown_connections 2024-06-02T15:50:24.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:24.815+0000 7fb13ba00700 1 -- 172.21.15.90:0/2154450486 wait complete. 2024-06-02T15:50:25.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:25 smithi120 ceph-mon[36451]: pgmap v693: 129 pgs: 129 active+clean; 90 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 542 op/s 2024-06-02T15:50:25.307 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:24 smithi120 podman[70085]: 2024-06-02 15:50:24.951908297 +0000 UTC m=+1.524341579 container create 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:50:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:25 smithi135 ceph-mon[36839]: pgmap v693: 129 pgs: 129 active+clean; 90 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 542 op/s 2024-06-02T15:50:25.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:25 smithi090 ceph-mon[31595]: pgmap v693: 129 pgs: 129 active+clean; 90 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 542 op/s 2024-06-02T15:50:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b58c00640 1 -- 172.21.15.90:0/4037170080 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fee245d2c0 con 0x55fee2584380 2024-06-02T15:50:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b58c00640 10 client.25955 handle_command_reply: tid=0 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 client.25955 shutdown 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 msgr2=0x55fee252aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c010970 tx=0x7f2b5c005970 comp rx=0 tx=0).stop 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 shutdown_connections 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fee2584380 0x55fee25867a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee252bae0 0x55fee252bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fee25122f0 0x55fee252aad0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 --2- 172.21.15.90:0/4037170080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fee24f1cf0 0x55fee252a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b58c00640 0 client.25955 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.061+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 >> 172.21.15.90:0/4037170080 conn(0x55fee24ed360 msgr2=0x55fee24ec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.062+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 shutdown_connections 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.062+0000 7f2b787ef740 1 -- 172.21.15.90:0/4037170080 wait complete. 2024-06-02T15:50:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.062+0000 7f2b787ef740 20 client.25955 trim_cache size 0 max 16384 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 msgr2=0x7f2b540b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 0x7f2b540b4bb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002a960 tx=0x7f2b60005ad0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 msgr2=0x7f2b7019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c420 tx=0x7f2b6800c8e0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 shutdown_connections 2024-06-02T15:50:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f2b540b26f0 0x7f2b540b4bb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2b7010eeb0 0x7f2b701a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2b70076a90 0x7f2b7019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 --2- 172.21.15.90:0/3920605956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b700754b0 0x7f2b7019f110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 >> 172.21.15.90:0/3920605956 conn(0x7f2b700fde50 msgr2=0x7f2b70102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 shutdown_connections 2024-06-02T15:50:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:26.065+0000 7f2b6fe00640 1 -- 172.21.15.90:0/3920605956 wait complete. 2024-06-02T15:50:27.458 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:27 smithi120 ceph-mon[36451]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 722 op/s 2024-06-02T15:50:27.458 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:27 smithi120 ceph-mon[36451]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-06-02T15:50:27.458 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:27 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 5 up:standby 2024-06-02T15:50:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:27 smithi135 ceph-mon[36839]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 722 op/s 2024-06-02T15:50:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:27 smithi135 ceph-mon[36839]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-06-02T15:50:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:27 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 5 up:standby 2024-06-02T15:50:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:27 smithi090 ceph-mon[31595]: pgmap v694: 129 pgs: 129 active+clean; 95 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 722 op/s 2024-06-02T15:50:27.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:27 smithi090 ceph-mon[31595]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-06-02T15:50:27.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:27 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 5 up:standby 2024-06-02T15:50:27.746 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:50:27.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:50:27.750 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:27 smithi120 podman[70085]: 2024-06-02 15:50:27.523343386 +0000 UTC m=+4.095776676 container init 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, RELEASE=main-b7328fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-06-02T15:50:27.750 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:27 smithi120 podman[70085]: 2024-06-02 15:50:27.527207288 +0000 UTC m=+4.099640587 container start 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, RELEASE=main-b7328fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:50:27.750 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:27 smithi120 ceph-mds[70108]: -- 172.21.15.120:0/2766608992 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a48e93a40 con 0x558a48e67000 2024-06-02T15:50:27.750 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[70102]: starting mds.h at 2024-06-02T15:50:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:29.057 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:28 smithi120 bash[70085]: 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad 2024-06-02T15:50:29.057 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 15:50:28 smithi120 systemd[1]: Started Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:50:29.497 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:29 smithi090 ceph-mon[31595]: pgmap v695: 129 pgs: 129 active+clean; 95 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 634 op/s 2024-06-02T15:50:29.497 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:29 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6836/1118882234,v1:172.21.15.120:6837/1118882234] up:boot 2024-06-02T15:50:29.497 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:29 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:50:29.497 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:29 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:50:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:29 smithi120 ceph-mon[36451]: pgmap v695: 129 pgs: 129 active+clean; 95 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 634 op/s 2024-06-02T15:50:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:29 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6836/1118882234,v1:172.21.15.120:6837/1118882234] up:boot 2024-06-02T15:50:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:29 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:50:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:29 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:50:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:29 smithi135 ceph-mon[36839]: pgmap v695: 129 pgs: 129 active+clean; 95 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 634 op/s 2024-06-02T15:50:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:29 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6836/1118882234,v1:172.21.15.120:6837/1118882234] up:boot 2024-06-02T15:50:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:29 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:50:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:29 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T15:50:30.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 -- 172.21.15.90:0/1827652127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c10da60 msgr2=0x7f280c10ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:30.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 --2- 172.21.15.90:0/1827652127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c10da60 0x7f280c10ff20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f280400b0a0 tx=0x7f280400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 -- 172.21.15.90:0/1827652127 shutdown_connections 2024-06-02T15:50:30.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 --2- 172.21.15.90:0/1827652127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c10da60 0x7f280c10ff20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 --2- 172.21.15.90:0/1827652127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f280c104630 0x7f280c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 --2- 172.21.15.90:0/1827652127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104f50 0x7f280c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 -- 172.21.15.90:0/1827652127 >> 172.21.15.90:0/1827652127 conn(0x7f280c0fde60 msgr2=0x7f280c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 -- 172.21.15.90:0/1827652127 shutdown_connections 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.529+0000 7f280be00700 1 -- 172.21.15.90:0/1827652127 wait complete. 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 Processor -- start 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 -- start start 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104630 0x7f280c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f280c10da60 0x7f280c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f280c1136b0 con 0x7f280c104630 2024-06-02T15:50:30.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.531+0000 7f280be00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f280c113530 con 0x7f280c104f50 2024-06-02T15:50:30.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.532+0000 7f280be00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f280c113830 con 0x7f280c10da60 2024-06-02T15:50:30.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.532+0000 7f280aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104630 0x7f280c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:30.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.532+0000 7f280a000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:30.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.532+0000 7f280a000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:34126/0 (socket says 172.21.15.90:34126) 2024-06-02T15:50:30.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.532+0000 7f280a000700 1 -- 172.21.15.90:0/2132527571 learned_addr learned my addr 172.21.15.90:0/2132527571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 -- 172.21.15.90:0/2132527571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f280c10da60 msgr2=0x7f280c1999b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f280c10da60 0x7f280c1999b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 -- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104630 msgr2=0x7f280c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104630 0x7f280c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 -- 172.21.15.90:0/2132527571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c19a110 con 0x7f280c104f50 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f280a000700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e980 tx=0x7f27f800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800cbd0 con 0x7f280c104f50 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f80044d0 con 0x7f280c104f50 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.534+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c19a400 con 0x7f280c104f50 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.533+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003df0 con 0x7f280c104f50 2024-06-02T15:50:30.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.534+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c1021c0 con 0x7f280c104f50 2024-06-02T15:50:30.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.535+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c06a230 con 0x7f280c104f50 2024-06-02T15:50:30.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.538+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f27f8003f90 con 0x7f280c104f50 2024-06-02T15:50:30.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.540+0000 7f2810a00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 0x7f27f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:30.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.540+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f27f80e2740 con 0x7f280c104f50 2024-06-02T15:50:30.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.540+0000 7f280aa00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 0x7f27f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:30.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.541+0000 7f280aa00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 0x7f27f00b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f280c101990 tx=0x7f27fc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:30.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.542+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27f80aa050 con 0x7f280c104f50 2024-06-02T15:50:30.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.980+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f280c02e560 con 0x7f280c104f50 2024-06-02T15:50:30.979 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:50:30.979 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:50:26.737457+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:50:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.981+0000 7f2810a00700 1 -- 172.21.15.90:0/2132527571 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11884 (secure 0 0 0) 0x7f27f80a9950 con 0x7f280c104f50 2024-06-02T15:50:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 msgr2=0x7f27f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 0x7f27f00b4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f280c101990 tx=0x7f27fc0073d0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 msgr2=0x7f280c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e980 tx=0x7f27f800ec90 comp rx=0 tx=0).stop 2024-06-02T15:50:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 shutdown_connections 2024-06-02T15:50:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27f00b2880 0x7f27f00b4d40 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f280c101990 tx=0x7f27fc0073d0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f280c10da60 0x7f280c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f280c104630 0x7f280c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 --2- 172.21.15.90:0/2132527571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f280c104f50 0x7f280c19fce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.984+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 >> 172.21.15.90:0/2132527571 conn(0x7f280c0fde60 msgr2=0x7f280c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.985+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 shutdown_connections 2024-06-02T15:50:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:30.985+0000 7f280be00700 1 -- 172.21.15.90:0/2132527571 wait complete. 2024-06-02T15:50:30.987 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-06-02T15:50:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:31 smithi090 ceph-mon[31595]: pgmap v696: 129 pgs: 129 active+clean; 95 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 869 op/s 2024-06-02T15:50:31.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:31 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2132527571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:50:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:31 smithi120 ceph-mon[36451]: pgmap v696: 129 pgs: 129 active+clean; 95 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 869 op/s 2024-06-02T15:50:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:31 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2132527571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:50:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:31 smithi135 ceph-mon[36839]: pgmap v696: 129 pgs: 129 active+clean; 95 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 869 op/s 2024-06-02T15:50:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:31 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2132527571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:50:31.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:50:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:33.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:33 smithi090 ceph-mon[31595]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 874 op/s 2024-06-02T15:50:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:33 smithi120 ceph-mon[36451]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 874 op/s 2024-06-02T15:50:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:33 smithi135 ceph-mon[36839]: pgmap v697: 129 pgs: 129 active+clean; 95 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 874 op/s 2024-06-02T15:50:34.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 -- 172.21.15.90:0/1121722701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 msgr2=0x7f41fc06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 --2- 172.21.15.90:0/1121722701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc06b250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009990 tx=0x7f41ec009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 -- 172.21.15.90:0/1121722701 shutdown_connections 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 --2- 172.21.15.90:0/1121722701 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc105f80 0x7f41fc106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 --2- 172.21.15.90:0/1121722701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc06b250 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 --2- 172.21.15.90:0/1121722701 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc06b790 0x7f41fc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 -- 172.21.15.90:0/1121722701 >> 172.21.15.90:0/1121722701 conn(0x7f41fc0793f0 msgr2=0x7f41fc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 -- 172.21.15.90:0/1121722701 shutdown_connections 2024-06-02T15:50:34.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.611+0000 7f4202200700 1 -- 172.21.15.90:0/1121722701 wait complete. 2024-06-02T15:50:34.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.613+0000 7f4202200700 1 Processor -- start 2024-06-02T15:50:34.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 -- start start 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 0x7f41fc1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 0x7f41fc1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc119d20 con 0x7f41fc105f80 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc119ba0 con 0x7f41fc06b790 2024-06-02T15:50:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.614+0000 7f4202200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc119ea0 con 0x7f41fc108470 2024-06-02T15:50:34.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.615+0000 7f41faa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 0x7f41fc1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.615+0000 7f41faa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 0x7f41fc1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57306/0 (socket says 172.21.15.90:57306) 2024-06-02T15:50:34.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.615+0000 7f41fbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.615+0000 7f41faa00700 1 -- 172.21.15.90:0/2939810603 learned_addr learned my addr 172.21.15.90:0/2939810603 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:34.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.615+0000 7f41fb400700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 0x7f41fc1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fb400700 1 -- 172.21.15.90:0/2939810603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 msgr2=0x7f41fc1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fb400700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc1a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fb400700 1 -- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 msgr2=0x7f41fc1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fb400700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 0x7f41fc1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fb400700 1 -- 172.21.15.90:0/2939810603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1a5260 con 0x7f41fc06b790 2024-06-02T15:50:34.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.616+0000 7f41fbe00700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.617+0000 7f41fb400700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 0x7f41fc1a0250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098e0 tx=0x7f41ec0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.617+0000 7f41f3e00700 1 -- 172.21.15.90:0/2939810603 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec027070 con 0x7f41fc06b790 2024-06-02T15:50:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.617+0000 7f41f3e00700 1 -- 172.21.15.90:0/2939810603 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec022470 con 0x7f41fc06b790 2024-06-02T15:50:34.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.617+0000 7f41f3e00700 1 -- 172.21.15.90:0/2939810603 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec017650 con 0x7f41fc06b790 2024-06-02T15:50:34.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.617+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1a54f0 con 0x7f41fc06b790 2024-06-02T15:50:34.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.618+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1acdd0 con 0x7f41fc06b790 2024-06-02T15:50:34.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.624+0000 7f420cc46b80 1 Processor -- start 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.624+0000 7f420cc46b80 1 -- start start 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 0x5610d82139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d820db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610d816be40 con 0x5610d8213f30 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610d815b9e0 con 0x5610d8213610 2024-06-02T15:50:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f420cc46b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d815fa20 con 0x5610d820e140 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f41fbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 0x5610d82139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f41fbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 0x5610d82139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36002/0 (socket says 172.21.15.90:36002) 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f41fbe00700 1 -- 172.21.15.90:0/2704042913 learned_addr learned my addr 172.21.15.90:0/2704042913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.625+0000 7f41faa00700 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 -- 172.21.15.90:0/2704042913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 msgr2=0x5610d82139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 0x5610d82139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 -- 172.21.15.90:0/2704042913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 msgr2=0x5610d820db20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d820db20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 -- 172.21.15.90:0/2704042913 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d8124420 con 0x5610d820e140 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41f3e00700 1 -- 172.21.15.90:0/2939810603 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f41ec009d10 con 0x7f41fc06b790 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.626+0000 7f41faa00700 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820e5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41e800bc30 tx=0x7f41e800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86b3132b1294e124 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.627+0000 7f41f0c00700 1 -- 172.21.15.90:0/2704042913 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8015070 con 0x5610d820e140 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.627+0000 7f41f0c00700 1 -- 172.21.15.90:0/2704042913 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e800f970 con 0x5610d820e140 2024-06-02T15:50:34.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.627+0000 7f41f0c00700 1 -- 172.21.15.90:0/2704042913 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8003d50 con 0x5610d820e140 2024-06-02T15:50:34.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.628+0000 7f420cc46b80 1 -- 172.21.15.90:0/2704042913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 msgr2=0x5610d820e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.628+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820e5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41e800bc30 tx=0x7f41e800bf40 comp rx=0 tx=0).stop 2024-06-02T15:50:34.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.628+0000 7f41f3e00700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 0x7f41e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.629+0000 7f41f3e00700 1 -- 172.21.15.90:0/2939810603 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f41ec0e2ee0 con 0x7f41fc06b790 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f41fbe00700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 0x7f41e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f420cc46b80 1 -- 172.21.15.90:0/2704042913 shutdown_connections 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820e5c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213610 0x5610d82139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2704042913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d820db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.630+0000 7f420cc46b80 1 -- 172.21.15.90:0/2704042913 >> 172.21.15.90:0/2704042913 conn(0x5610d813bbe0 msgr2=0x5610d8208690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.631+0000 7f420cc46b80 1 -- 172.21.15.90:0/2704042913 shutdown_connections 2024-06-02T15:50:34.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.632+0000 7f420cc46b80 1 -- 172.21.15.90:0/2704042913 wait complete. 2024-06-02T15:50:34.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.633+0000 7f420cc46b80 1 -- start start 2024-06-02T15:50:34.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.633+0000 7f420cc46b80 1 Processor -- start 2024-06-02T15:50:34.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.633+0000 7f41fbe00700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 0x7f41e00b4dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009fd0 tx=0x7f41f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 0x5610d8217000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 0x5610d8217540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610d8061fa0 con 0x5610d820e140 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610d80fc760 con 0x5610d8213f30 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d821a9e0 con 0x5610d8213610 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.634+0000 7f420cc46b80 10 client.0 _ll_register_callbacks cb 0x5610d8105c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 0x5610d8217540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57310/0 (socket says 172.21.15.90:57310) 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fb400700 1 -- 172.21.15.90:0/2614183275 learned_addr learned my addr 172.21.15.90:0/2614183275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:34.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 0x5610d8217000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 -- 172.21.15.90:0/2614183275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 msgr2=0x5610d8217540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 0x5610d8217540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 -- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 msgr2=0x5610d8219ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41faa00700 1 -- 172.21.15.90:0/2614183275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d80ab8d0 con 0x5610d8213610 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fb400700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.635+0000 7f41fbe00700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 0x5610d8217540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.638+0000 7f41e5600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:34.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.638+0000 7f41faa00700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 0x5610d8217000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41e800bc30 tx=0x7f41e800fd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755a94c646525580 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.639+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8015070 con 0x5610d8213610 2024-06-02T15:50:34.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.639+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e800f970 con 0x5610d8213610 2024-06-02T15:50:34.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.639+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f41e8005020 con 0x5610d8213610 2024-06-02T15:50:34.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.639+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f41e8005220 con 0x5610d8213610 2024-06-02T15:50:34.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.640+0000 7f420cc46b80 10 client.25939 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:34.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.640+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610d812c060 con 0x5610d8213610 2024-06-02T15:50:34.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.640+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41e800f970 con 0x5610d8213610 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f41e8011650 con 0x5610d8213610 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 10 client.25939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 20 client.25939 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d82274f0 0x5610d82298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 4 client.25939 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d816dd90 con 0x5610d82274f0 2024-06-02T15:50:34.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.641+0000 7f41fb400700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d82274f0 0x5610d82298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.643+0000 7f41e5600700 10 client.25939 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:50:34.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.643+0000 7f41fb400700 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d82274f0 0x5610d82298e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52b4fc44856954bc server_cookie=d4d1a3c6cb6b9750 in_seq=0 out_seq=0 2024-06-02T15:50:34.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.644+0000 7f41e5600700 1 -- 172.21.15.90:0/2614183275 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5610d816dd90 con 0x5610d82274f0 2024-06-02T15:50:34.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.644+0000 7f41e5600700 10 client.25939 handle_command_reply: tid=0 2024-06-02T15:50:34.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.644+0000 7f420cc46b80 1 client.25939 shutdown 2024-06-02T15:50:34.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 msgr2=0x5610d8217000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 0x5610d8217000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f41e800bc30 tx=0x7f41e800fd30 comp rx=0 tx=0).stop 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 shutdown_connections 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d82274f0 0x5610d82298e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d8213f30 0x5610d8217540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d820e140 0x5610d8219ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 --2- 172.21.15.90:0/2614183275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d8213610 0x5610d8217000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.645+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 >> 172.21.15.90:0/2614183275 conn(0x5610d813bbe0 msgr2=0x5610d8207d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.646+0000 7f41e5600700 0 client.25939 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:50:34.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.646+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 shutdown_connections 2024-06-02T15:50:34.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.646+0000 7f420cc46b80 1 -- 172.21.15.90:0/2614183275 wait complete. 2024-06-02T15:50:34.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.647+0000 7f420cc46b80 20 client.25939 trim_cache size 0 max 16384 2024-06-02T15:50:34.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.666+0000 7f420cc46b80 1 Processor -- start 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 -- start start 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d8223ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e540 0x5610d8215f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b81b0 con 0x5610d8213f30 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b59e0 con 0x5610d820e540 2024-06-02T15:50:34.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.667+0000 7f420cc46b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b3830 con 0x5610d820e140 2024-06-02T15:50:34.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.668+0000 7f41fb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.668+0000 7f41fb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35102/0 (socket says 172.21.15.90:35102) 2024-06-02T15:50:34.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.669+0000 7f41fb400700 1 -- 172.21.15.90:0/1060867445 learned_addr learned my addr 172.21.15.90:0/1060867445 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:34.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.669+0000 7f41fbe00700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e540 0x5610d8215f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.669+0000 7f41fb400700 1 -- 172.21.15.90:0/1060867445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e540 msgr2=0x5610d8215f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.669+0000 7f41fb400700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e540 0x5610d8215f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.669+0000 7f41fb400700 1 -- 172.21.15.90:0/1060867445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 msgr2=0x5610d8223ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.670+0000 7f41faa00700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d8223ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.670+0000 7f41fb400700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d8223ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.670+0000 7f41fb400700 1 -- 172.21.15.90:0/1060867445 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d82298e0 con 0x5610d820e140 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.670+0000 7f41faa00700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d8223ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:34.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.671+0000 7f41fb400700 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0a6260 tx=0x7f41ec009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ea6e03180f6e65f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.671+0000 7f41e7e00700 1 -- 172.21.15.90:0/1060867445 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0c3020 con 0x5610d820e140 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.671+0000 7f41e7e00700 1 -- 172.21.15.90:0/1060867445 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec0a9cf0 con 0x5610d820e140 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.671+0000 7f41e7e00700 1 -- 172.21.15.90:0/1060867445 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec022620 con 0x5610d820e140 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.672+0000 7f420cc46b80 1 -- 172.21.15.90:0/1060867445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 msgr2=0x5610d820db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.672+0000 7f420cc46b80 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0a6260 tx=0x7f41ec009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.672+0000 7f420cc46b80 1 -- 172.21.15.90:0/1060867445 shutdown_connections 2024-06-02T15:50:34.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.672+0000 7f420cc46b80 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e540 0x5610d8215f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.673+0000 7f420cc46b80 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e140 0x5610d820db20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.673+0000 7f420cc46b80 1 --2- 172.21.15.90:0/1060867445 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d8223ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.673+0000 7f420cc46b80 1 -- 172.21.15.90:0/1060867445 >> 172.21.15.90:0/1060867445 conn(0x5610d813bbe0 msgr2=0x5610d8214370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.673+0000 7f420cc46b80 1 -- 172.21.15.90:0/1060867445 shutdown_connections 2024-06-02T15:50:34.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.673+0000 7f420cc46b80 1 -- 172.21.15.90:0/1060867445 wait complete. 2024-06-02T15:50:34.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.674+0000 7f420cc46b80 1 -- start start 2024-06-02T15:50:34.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.674+0000 7f420cc46b80 1 Processor -- start 2024-06-02T15:50:34.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.675+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.675+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.675+0000 7f420cc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d822f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41faa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41fb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41fb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36024/0 (socket says 172.21.15.90:36024) 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41faa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35110/0 (socket says 172.21.15.90:35110) 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41fb400700 1 -- 172.21.15.90:0/587955582 learned_addr learned my addr 172.21.15.90:0/587955582 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:34.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f41fbe00700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d822f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b3ea0 con 0x5610d8213f30 2024-06-02T15:50:34.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b5310 con 0x5610d820e140 2024-06-02T15:50:34.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b4570 con 0x5610d820e540 2024-06-02T15:50:34.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.676+0000 7f420cc46b80 10 client.0 _ll_register_callbacks cb 0x5610d8105c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:34.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41faa00700 1 -- 172.21.15.90:0/587955582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 msgr2=0x5610d822ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41faa00700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41faa00700 1 -- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 msgr2=0x5610d822f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41faa00700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d822f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41faa00700 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d8290ea0 con 0x5610d820e540 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41fb400700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:34.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.677+0000 7f41fbe00700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d822f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.678+0000 7f41faa00700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5610d8218090 tx=0x7f41e800fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9289969ec72b55 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.678+0000 7f41e5600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.678+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8015040 con 0x5610d820e540 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.678+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f41e8005650 con 0x5610d820e540 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.678+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec0a9cf0 con 0x5610d820e540 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.679+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f41e8004030 con 0x5610d820e540 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.679+0000 7f420cc46b80 10 client.25945 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:34.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.679+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610d82920b0 con 0x5610d820e540 2024-06-02T15:50:34.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec0a9cf0 con 0x5610d820e540 2024-06-02T15:50:34.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f41e8004710 con 0x5610d820e540 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 10 client.25945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 20 client.25945 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d8292760 0x5610d8294b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 4 client.25945 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.680+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610d8167570 con 0x5610d8292760 2024-06-02T15:50:34.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.681+0000 7f41fb400700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d8292760 0x5610d8294b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:34.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.682+0000 7f41fb400700 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d8292760 0x5610d8294b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62310464776c5f84 server_cookie=b76385db7046cd40 in_seq=0 out_seq=0 2024-06-02T15:50:34.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.682+0000 7f41e5600700 10 client.25945 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:50:34.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.685+0000 7f41e5600700 1 -- 172.21.15.90:0/587955582 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610d8167570 con 0x5610d8292760 2024-06-02T15:50:34.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.686+0000 7f41e5600700 10 client.25945 handle_command_reply: tid=0 2024-06-02T15:50:34.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.686+0000 7f420cc46b80 1 client.25945 shutdown 2024-06-02T15:50:34.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.686+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 msgr2=0x5610d822efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.686+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x5610d8218090 tx=0x7f41e800fa40 comp rx=0 tx=0).stop 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 shutdown_connections 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5610d8292760 0x5610d8294b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d8213f30 0x5610d822f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d820e140 0x5610d822ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 --2- 172.21.15.90:0/587955582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d820e540 0x5610d822efd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 >> 172.21.15.90:0/587955582 conn(0x5610d813bbe0 msgr2=0x5610d82174f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f41e5600700 0 client.25945 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 shutdown_connections 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 1 -- 172.21.15.90:0/587955582 wait complete. 2024-06-02T15:50:34.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.687+0000 7f420cc46b80 20 client.25945 trim_cache size 0 max 16384 2024-06-02T15:50:34.686 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:50:34.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 msgr2=0x7f41e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 0x7f41e00b4dc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009fd0 tx=0x7f41f400b040 comp rx=0 tx=0).stop 2024-06-02T15:50:34.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 msgr2=0x7f41fc1a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:34.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 0x7f41fc1a0250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0098e0 tx=0x7f41ec0057c0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 shutdown_connections 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f41e00b2900 0x7f41e00b4dc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41fc108470 0x7f41fc1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f41fc06b790 0x7f41fc1a0250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 --2- 172.21.15.90:0/2939810603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41fc105f80 0x7f41fc1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:34.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.689+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 >> 172.21.15.90:0/2939810603 conn(0x7f41fc0793f0 msgr2=0x7f41fc101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:34.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.692+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 shutdown_connections 2024-06-02T15:50:34.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:34.693+0000 7f4202200700 1 -- 172.21.15.90:0/2939810603 wait complete. 2024-06-02T15:50:35.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:35 smithi090 ceph-mon[31595]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 847 op/s 2024-06-02T15:50:35.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:50:35.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:35 smithi120 ceph-mon[36451]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 847 op/s 2024-06-02T15:50:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:35 smithi135 ceph-mon[36839]: pgmap v698: 129 pgs: 129 active+clean; 95 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 847 op/s 2024-06-02T15:50:36.056 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:37.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:37 smithi090 ceph-mon[31595]: pgmap v699: 129 pgs: 129 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 856 op/s 2024-06-02T15:50:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:37 smithi120 ceph-mon[36451]: pgmap v699: 129 pgs: 129 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 856 op/s 2024-06-02T15:50:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:37 smithi135 ceph-mon[36839]: pgmap v699: 129 pgs: 129 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 856 op/s 2024-06-02T15:50:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.363+0000 7f3093400700 1 -- 172.21.15.90:0/2883894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c06b220 msgr2=0x7f308c06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.363+0000 7f3093400700 1 --2- 172.21.15.90:0/2883894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c06b220 0x7f308c06b6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3078009990 tx=0x7f3078009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 -- 172.21.15.90:0/2883894 shutdown_connections 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 --2- 172.21.15.90:0/2883894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 --2- 172.21.15.90:0/2883894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c1095b0 0x7f308c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 --2- 172.21.15.90:0/2883894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c06b220 0x7f308c06b6a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 -- 172.21.15.90:0/2883894 >> 172.21.15.90:0/2883894 conn(0x7f308c0793f0 msgr2=0x7f308c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 -- 172.21.15.90:0/2883894 shutdown_connections 2024-06-02T15:50:38.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.366+0000 7f3093400700 1 -- 172.21.15.90:0/2883894 wait complete. 2024-06-02T15:50:38.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 Processor -- start 2024-06-02T15:50:38.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 -- start start 2024-06-02T15:50:38.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c06b220 0x7f308c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 0x7f308c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f308c111590 con 0x7f308c1095b0 2024-06-02T15:50:38.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f308c111410 con 0x7f308c06b220 2024-06-02T15:50:38.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.367+0000 7f3093400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f308c111710 con 0x7f308c06bbe0 2024-06-02T15:50:38.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.368+0000 7f308be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.368+0000 7f308be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35122/0 (socket says 172.21.15.90:35122) 2024-06-02T15:50:38.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.368+0000 7f308be00700 1 -- 172.21.15.90:0/164436716 learned_addr learned my addr 172.21.15.90:0/164436716 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:38.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.368+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 0x7f308c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.368+0000 7f3090a00700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c06b220 0x7f308c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 -- 172.21.15.90:0/164436716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 msgr2=0x7f308c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 -- 172.21.15.90:0/164436716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c06b220 msgr2=0x7f308c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c06b220 0x7f308c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 -- 172.21.15.90:0/164436716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c198030 con 0x7f308c1095b0 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 0x7f308c197850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f308400efc0 tx=0x7f308400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3089600700 1 -- 172.21.15.90:0/164436716 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840092e0 con 0x7f308c1095b0 2024-06-02T15:50:38.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c1982c0 con 0x7f308c1095b0 2024-06-02T15:50:38.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.369+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1a4530 con 0x7f308c1095b0 2024-06-02T15:50:38.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.370+0000 7f3089600700 1 -- 172.21.15.90:0/164436716 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308400c9f0 con 0x7f308c1095b0 2024-06-02T15:50:38.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.370+0000 7f3089600700 1 -- 172.21.15.90:0/164436716 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084004440 con 0x7f308c1095b0 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.375+0000 7f3089600700 1 -- 172.21.15.90:0/164436716 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f308401b030 con 0x7f308c1095b0 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.375+0000 7f309dda6b80 1 Processor -- start 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.375+0000 7f309dda6b80 1 -- start start 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a6fde40 con 0x55bd6a7b0450 2024-06-02T15:50:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a6ed9e0 con 0x55bd6a7b0d70 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f309dda6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a6f1a20 con 0x55bd6a7b5080 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f308be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.376+0000 7f308be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57360/0 (socket says 172.21.15.90:57360) 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 -- 172.21.15.90:0/1595114777 learned_addr learned my addr 172.21.15.90:0/1595114777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f3089600700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 0x7f30740b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f3090a00700 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f3089600700 1 -- 172.21.15.90:0/164436716 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f30840e20b0 con 0x7f308c1095b0 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 -- 172.21.15.90:0/1595114777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b5080 msgr2=0x55bd6a7b54c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b54c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 -- 172.21.15.90:0/1595114777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b0d70 msgr2=0x55bd6a7b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.377+0000 7f308be00700 1 -- 172.21.15.90:0/1595114777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6a6b6420 con 0x55bd6a7b0450 2024-06-02T15:50:38.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.378+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 0x7f30740b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.378+0000 7f308be00700 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f30780096d0 tx=0x7f307800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1800603b224063 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.378+0000 7f307e000700 1 -- 172.21.15.90:0/1595114777 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780055d0 con 0x55bd6a7b0450 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.378+0000 7f307e000700 1 -- 172.21.15.90:0/1595114777 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3078051070 con 0x55bd6a7b0450 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.378+0000 7f307e000700 1 -- 172.21.15.90:0/1595114777 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078003b60 con 0x55bd6a7b0450 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.379+0000 7f309dda6b80 1 -- 172.21.15.90:0/1595114777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 msgr2=0x55bd6a7b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.379+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f30780096d0 tx=0x7f307800b260 comp rx=0 tx=0).stop 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 -- 172.21.15.90:0/1595114777 shutdown_connections 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7b0830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1595114777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 -- 172.21.15.90:0/1595114777 >> 172.21.15.90:0/1595114777 conn(0x55bd6a6cdbe0 msgr2=0x55bd6a79a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 -- 172.21.15.90:0/1595114777 shutdown_connections 2024-06-02T15:50:38.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f3091400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 0x7f30740b4e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f3084009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.380+0000 7f309dda6b80 1 -- 172.21.15.90:0/1595114777 wait complete. 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.381+0000 7f309dda6b80 1 -- start start 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.381+0000 7f309dda6b80 1 Processor -- start 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a5f3fa0 con 0x55bd6a7b0450 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a68e760 con 0x55bd6a7b5080 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd6a79f440 con 0x55bd6a7b0d70 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.382+0000 7f309dda6b80 10 client.0 _ll_register_callbacks cb 0x55bd6a697c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.383+0000 7f3090a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.383+0000 7f3090a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35132/0 (socket says 172.21.15.90:35132) 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.383+0000 7f3090a00700 1 -- 172.21.15.90:0/1554576992 learned_addr learned my addr 172.21.15.90:0/1554576992 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.383+0000 7f308be00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.383+0000 7f308be00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 -- 172.21.15.90:0/1554576992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 msgr2=0x55bd6a7b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 -- 172.21.15.90:0/1554576992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 msgr2=0x55bd6a7b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 -- 172.21.15.90:0/1554576992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6a63d8d0 con 0x55bd6a7b0450 2024-06-02T15:50:38.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f308be00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bc4a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3078006010 tx=0x7f3078000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf4e681d0bfd8fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.384+0000 7f3072a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:50:38.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.385+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078063060 con 0x55bd6a7b0450 2024-06-02T15:50:38.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.385+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3078003b30 con 0x55bd6a7b0450 2024-06-02T15:50:38.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.385+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3078051070 con 0x55bd6a7b0450 2024-06-02T15:50:38.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.386+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f307806a050 con 0x55bd6a7b0450 2024-06-02T15:50:38.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.386+0000 7f309dda6b80 10 client.15825 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:38.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.386+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd6a6be060 con 0x55bd6a7b0450 2024-06-02T15:50:38.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.386+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3078051070 con 0x55bd6a7b0450 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f307800fc80 con 0x55bd6a7b0450 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 20 client.15825 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a7abe70 0x55bd6a7ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 4 client.15825 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd6a6ffd90 con 0x55bd6a7abe70 2024-06-02T15:50:38.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.387+0000 7f3090a00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a7abe70 0x55bd6a7ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.388+0000 7f3090a00700 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a7abe70 0x55bd6a7ae290 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dbc8c4ed44dbe76 server_cookie=ac15a16dcfdeb181 in_seq=0 out_seq=0 2024-06-02T15:50:38.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.388+0000 7f3072a00700 10 client.15825 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:50:38.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.389+0000 7f3072a00700 1 -- 172.21.15.90:0/1554576992 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bd6a6ffd90 con 0x55bd6a7abe70 2024-06-02T15:50:38.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.389+0000 7f3072a00700 10 client.15825 handle_command_reply: tid=0 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 client.15825 shutdown 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 msgr2=0x55bd6a7bc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bc4a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3078006010 tx=0x7f3078000bc0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 shutdown_connections 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a7abe70 0x55bd6a7ae290 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bc4a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 --2- 172.21.15.90:0/1554576992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.390+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 >> 172.21.15.90:0/1554576992 conn(0x55bd6a6cdbe0 msgr2=0x55bd6a79e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.391+0000 7f3072a00700 0 client.15825 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:50:38.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.391+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 shutdown_connections 2024-06-02T15:50:38.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.392+0000 7f309dda6b80 1 -- 172.21.15.90:0/1554576992 wait complete. 2024-06-02T15:50:38.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.392+0000 7f309dda6b80 20 client.15825 trim_cache size 0 max 16384 2024-06-02T15:50:38.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 Processor -- start 2024-06-02T15:50:38.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 -- start start 2024-06-02T15:50:38.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f306c011540 con 0x55bd6a7b0450 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f306c00ed70 con 0x55bd6a7b5080 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.415+0000 7f309dda6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f306c00cbc0 con 0x55bd6a7b0d70 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35138/0 (socket says 172.21.15.90:35138) 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 -- 172.21.15.90:0/2446239393 learned_addr learned my addr 172.21.15.90:0/2446239393 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 -- 172.21.15.90:0/2446239393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 msgr2=0x55bd6a7b9b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 -- 172.21.15.90:0/2446239393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 msgr2=0x55bd6a7ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.416+0000 7f3090a00700 1 -- 172.21.15.90:0/2446239393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd6a7b4e90 con 0x55bd6a7b0d70 2024-06-02T15:50:38.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.417+0000 7f3090a00700 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55bd6a6fded0 tx=0x7f3080011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22e4b88920b6281 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.418+0000 7f307d600700 1 -- 172.21.15.90:0/2446239393 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080011cf0 con 0x55bd6a7b0d70 2024-06-02T15:50:38.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.418+0000 7f307d600700 1 -- 172.21.15.90:0/2446239393 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308001d8e0 con 0x55bd6a7b0d70 2024-06-02T15:50:38.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.418+0000 7f307d600700 1 -- 172.21.15.90:0/2446239393 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080010b20 con 0x55bd6a7b0d70 2024-06-02T15:50:38.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 -- 172.21.15.90:0/2446239393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 msgr2=0x55bd6a7b9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55bd6a6fded0 tx=0x7f3080011720 comp rx=0 tx=0).stop 2024-06-02T15:50:38.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 -- 172.21.15.90:0/2446239393 shutdown_connections 2024-06-02T15:50:38.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0450 0x55bd6a7ba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2446239393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7b9620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 -- 172.21.15.90:0/2446239393 >> 172.21.15.90:0/2446239393 conn(0x55bd6a6cdbe0 msgr2=0x55bd6a7b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.419+0000 7f309dda6b80 1 -- 172.21.15.90:0/2446239393 shutdown_connections 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.420+0000 7f309dda6b80 1 -- 172.21.15.90:0/2446239393 wait complete. 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.420+0000 7f309dda6b80 1 -- start start 2024-06-02T15:50:38.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.420+0000 7f309dda6b80 1 Processor -- start 2024-06-02T15:50:38.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f306c00d230 con 0x55bd6a7b0d70 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f306c00e6a0 con 0x55bd6a7b5080 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f306c00d900 con 0x55bd6a7b0450 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.421+0000 7f309dda6b80 10 client.0 _ll_register_callbacks cb 0x55bd6a697c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.422+0000 7f3090a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.422+0000 7f3090a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57378/0 (socket says 172.21.15.90:57378) 2024-06-02T15:50:38.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.422+0000 7f3090a00700 1 -- 172.21.15.90:0/2486559061 learned_addr learned my addr 172.21.15.90:0/2486559061 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:38.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 -- 172.21.15.90:0/2486559061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0450 msgr2=0x55bd6a7bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 -- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 msgr2=0x55bd6a7c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f308be00700 1 -- 172.21.15.90:0/2486559061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd6a821fe0 con 0x55bd6a7b5080 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.423+0000 7f3090a00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.424+0000 7f308be00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7c0880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3078005ac0 tx=0x7f3078009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc655b37930f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:38.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.424+0000 7f3072a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:38.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.425+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078063040 con 0x55bd6a7b5080 2024-06-02T15:50:38.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.425+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f3078055c90 con 0x55bd6a7b5080 2024-06-02T15:50:38.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.425+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f308001d8e0 con 0x55bd6a7b5080 2024-06-02T15:50:38.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.425+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3078055e90 con 0x55bd6a7b5080 2024-06-02T15:50:38.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.427+0000 7f309dda6b80 10 client.26015 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:38.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.427+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd6a8232a0 con 0x55bd6a7b5080 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308001d8e0 con 0x55bd6a7b5080 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f307806e660 con 0x55bd6a7b5080 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 10 client.26015 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 10 client.26015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 20 client.26015 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:38.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a8238a0 0x55bd6a825cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:38.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 4 client.26015 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:50:38.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.428+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd6a6f9570 con 0x55bd6a8238a0 2024-06-02T15:50:38.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.429+0000 7f3090a00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a8238a0 0x55bd6a825cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:38.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.431+0000 7f3090a00700 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a8238a0 0x55bd6a825cc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=674e7aa78574dd21 server_cookie=51539a0349fba988 in_seq=0 out_seq=0 2024-06-02T15:50:38.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.431+0000 7f3072a00700 10 client.26015 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:50:38.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f3072a00700 1 -- 172.21.15.90:0/2486559061 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd6a6f9570 con 0x55bd6a8238a0 2024-06-02T15:50:38.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f3072a00700 10 client.26015 handle_command_reply: tid=0 2024-06-02T15:50:38.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 client.26015 shutdown 2024-06-02T15:50:38.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 msgr2=0x55bd6a7c0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7c0880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3078005ac0 tx=0x7f3078009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 shutdown_connections 2024-06-02T15:50:38.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55bd6a8238a0 0x55bd6a825cc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd6a7b5080 0x55bd6a7c0880 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd6a7b0450 0x55bd6a7bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 --2- 172.21.15.90:0/2486559061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd6a7b0d70 0x55bd6a7c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 >> 172.21.15.90:0/2486559061 conn(0x55bd6a6cdbe0 msgr2=0x55bd6a7b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.432+0000 7f3072a00700 0 client.26015 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.433+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 shutdown_connections 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.433+0000 7f309dda6b80 1 -- 172.21.15.90:0/2486559061 wait complete. 2024-06-02T15:50:38.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.433+0000 7f309dda6b80 20 client.26015 trim_cache size 0 max 16384 2024-06-02T15:50:38.431 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:50:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 msgr2=0x7f30740b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 0x7f30740b4e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f3084009480 comp rx=0 tx=0).stop 2024-06-02T15:50:38.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 msgr2=0x7f308c197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 0x7f308c197850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f308400efc0 tx=0x7f308400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 shutdown_connections 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f30740b2970 0x7f30740b4e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f308c1095b0 0x7f308c197850 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f308c06b220 0x7f308c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 --2- 172.21.15.90:0/164436716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f308c06bbe0 0x7f308c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.436+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 >> 172.21.15.90:0/164436716 conn(0x7f308c0793f0 msgr2=0x7f308c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.437+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 shutdown_connections 2024-06-02T15:50:38.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:38.437+0000 7f3093400700 1 -- 172.21.15.90:0/164436716 wait complete. 2024-06-02T15:50:39.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:50:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.245+0000 7f1a47e00640 1 -- 172.21.15.90:0/3689185470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a48110d90 msgr2=0x7f1a48113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.245+0000 7f1a47e00640 1 --2- 172.21.15.90:0/3689185470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a48110d90 0x7f1a48113180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009be0 tx=0x7f1a4002a6d0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.245+0000 7f1a47e00640 1 -- 172.21.15.90:0/3689185470 shutdown_connections 2024-06-02T15:50:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.246+0000 7f1a47e00640 1 --2- 172.21.15.90:0/3689185470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a48110d90 0x7f1a48113180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.246+0000 7f1a47e00640 1 --2- 172.21.15.90:0/3689185470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a48074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.246+0000 7f1a47e00640 1 --2- 172.21.15.90:0/3689185470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a4810a660 0x7f1a48073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.246+0000 7f1a47e00640 1 -- 172.21.15.90:0/3689185470 >> 172.21.15.90:0/3689185470 conn(0x7f1a4806f8f0 msgr2=0x7f1a48071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.247+0000 7f1a47e00640 1 -- 172.21.15.90:0/3689185470 shutdown_connections 2024-06-02T15:50:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.247+0000 7f1a47e00640 1 -- 172.21.15.90:0/3689185470 wait complete. 2024-06-02T15:50:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.248+0000 7f1a47e00640 1 Processor -- start 2024-06-02T15:50:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.248+0000 7f1a47e00640 1 -- start start 2024-06-02T15:50:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a4810a660 0x7f1a4810ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 0x7f1a4810b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48116bd0 con 0x7f1a48110d90 2024-06-02T15:50:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48116a50 con 0x7f1a4810a660 2024-06-02T15:50:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48116d50 con 0x7f1a48073fc0 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a46a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a46a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35170/0 (socket says 172.21.15.90:35170) 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a46a00640 1 -- 172.21.15.90:0/1631675783 learned_addr learned my addr 172.21.15.90:0/1631675783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.249+0000 7f1a47400640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 0x7f1a4810b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 -- 172.21.15.90:0/1631675783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a4810a660 msgr2=0x7f1a4810ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46000640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a4810a660 0x7f1a4810ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a4810a660 0x7f1a4810ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 -- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 msgr2=0x7f1a4810b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 0x7f1a4810b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 -- 172.21.15.90:0/1631675783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4810baf0 con 0x7f1a48073fc0 2024-06-02T15:50:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a47400640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 0x7f1a4810b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.250+0000 7f1a46a00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800d950 tx=0x7f1a3800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.251+0000 7f1a3f400640 1 -- 172.21.15.90:0/1631675783 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38017070 con 0x7f1a48073fc0 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.251+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a481ab3a0 con 0x7f1a48073fc0 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.251+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481ab940 con 0x7f1a48073fc0 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.251+0000 7f1a3f400640 1 -- 172.21.15.90:0/1631675783 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a38009c30 con 0x7f1a48073fc0 2024-06-02T15:50:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.251+0000 7f1a3f400640 1 -- 172.21.15.90:0/1631675783 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800f620 con 0x7f1a48073fc0 2024-06-02T15:50:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 Processor -- start 2024-06-02T15:50:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 -- start start 2024-06-02T15:50:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db136e0 0x56283db181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db1ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56283da82b90 con 0x56283db136e0 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56283da72730 con 0x56283db12cf0 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.255+0000 7f1a5085c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56283da76770 con 0x56283db18850 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a47400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a47400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36112/0 (socket says 172.21.15.90:36112) 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a47400640 1 -- 172.21.15.90:0/490447128 learned_addr learned my addr 172.21.15.90:0/490447128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a46000640 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db1ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a47400640 1 -- 172.21.15.90:0/490447128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 msgr2=0x56283db1ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.256+0000 7f1a47400640 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db1ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.257+0000 7f1a47400640 1 -- 172.21.15.90:0/490447128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db136e0 msgr2=0x56283db181e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.257+0000 7f1a47400640 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db136e0 0x56283db181e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.259+0000 7f1a47400640 1 -- 172.21.15.90:0/490447128 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283da0b8b0 con 0x56283db12cf0 2024-06-02T15:50:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.259+0000 7f1a46000640 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.260+0000 7f1a3f400640 1 -- 172.21.15.90:0/1631675783 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1a38010460 con 0x7f1a48073fc0 2024-06-02T15:50:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.260+0000 7f1a47400640 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a40004af0 tx=0x7f1a40005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136de2d0d8d4d8ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.260+0000 7f1a33e00640 1 -- 172.21.15.90:0/490447128 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40041030 con 0x56283db12cf0 2024-06-02T15:50:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.260+0000 7f1a33e00640 1 -- 172.21.15.90:0/490447128 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x56283db12cf0 2024-06-02T15:50:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.260+0000 7f1a33e00640 1 -- 172.21.15.90:0/490447128 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40036040 con 0x56283db12cf0 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.261+0000 7f1a5085c740 1 -- 172.21.15.90:0/490447128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 msgr2=0x56283db13110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.261+0000 7f1a5085c740 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1a40004af0 tx=0x7f1a40005ec0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a3f400640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 0x7f1a2c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a3f400640 1 -- 172.21.15.90:0/1631675783 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1a380e2460 con 0x7f1a48073fc0 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a47400640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 0x7f1a2c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a5085c740 1 -- 172.21.15.90:0/490447128 shutdown_connections 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a5085c740 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db1ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a5085c740 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db136e0 0x56283db181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a5085c740 1 --2- 172.21.15.90:0/490447128 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db12cf0 0x56283db13110 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.262+0000 7f1a5085c740 1 -- 172.21.15.90:0/490447128 >> 172.21.15.90:0/490447128 conn(0x56283db0e360 msgr2=0x56283db0e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.263+0000 7f1a47400640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 0x7f1a2c0b4da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009560 tx=0x7f1a40049040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.263+0000 7f1a5085c740 1 -- 172.21.15.90:0/490447128 shutdown_connections 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.263+0000 7f1a5085c740 1 -- 172.21.15.90:0/490447128 wait complete. 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.264+0000 7f1a5085c740 1 -- start start 2024-06-02T15:50:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.264+0000 7f1a5085c740 1 Processor -- start 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db1cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db1d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56283d758790 con 0x56283db12cf0 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x56283da1abc0 con 0x56283db136e0 2024-06-02T15:50:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.265+0000 7f1a5085c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56283db1b7a0 con 0x56283db18850 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35176/0 (socket says 172.21.15.90:35176) 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 -- 172.21.15.90:0/798049598 learned_addr learned my addr 172.21.15.90:0/798049598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a46a00640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db1cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a5085c740 10 client.0 _ll_register_callbacks cb 0x56283d9f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a46000640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 -- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 msgr2=0x56283db1d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db1d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 -- 172.21.15.90:0/798049598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 msgr2=0x56283db1cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db1cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.266+0000 7f1a47400640 1 -- 172.21.15.90:0/798049598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283d787210 con 0x56283db18850 2024-06-02T15:50:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.267+0000 7f1a47400640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a40034040 tx=0x7f1a4002a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29aea5e06b6e28 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.267+0000 7f1a30c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.267+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40057070 con 0x56283db18850 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.267+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1a40004730 con 0x56283db18850 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.267+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40003970 con 0x56283db18850 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.268+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a40041030 con 0x56283db18850 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.269+0000 7f1a5085c740 10 client.25972 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.269+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56283d853620 con 0x56283db18850 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x56283db18850 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1a40050050 con 0x56283db18850 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 10 client.25972 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 20 client.25972 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db28310 0x56283db2a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 4 client.25972 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:39.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.270+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283da84ae0 con 0x56283db28310 2024-06-02T15:50:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.271+0000 7f1a46a00640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db28310 0x56283db2a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.272+0000 7f1a46a00640 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db28310 0x56283db2a740 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc4222e18312422 server_cookie=a6c8ee3887fe7e1a in_seq=0 out_seq=0 2024-06-02T15:50:39.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.272+0000 7f1a30c00640 10 client.25972 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.272+0000 7f1a30c00640 1 -- 172.21.15.90:0/798049598 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56283da84ae0 con 0x56283db28310 2024-06-02T15:50:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.272+0000 7f1a30c00640 10 client.25972 handle_command_reply: tid=0 2024-06-02T15:50:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.272+0000 7f1a5085c740 1 client.25972 shutdown 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 msgr2=0x56283db240c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a40034040 tx=0x7f1a4002a6d0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 shutdown_connections 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db28310 0x56283db2a740 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db240c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db1d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 --2- 172.21.15.90:0/798049598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db1cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 >> 172.21.15.90:0/798049598 conn(0x56283db0e360 msgr2=0x56283db0d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a30c00640 0 client.25972 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 shutdown_connections 2024-06-02T15:50:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.273+0000 7f1a5085c740 1 -- 172.21.15.90:0/798049598 wait complete. 2024-06-02T15:50:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.274+0000 7f1a5085c740 20 client.25972 trim_cache size 0 max 16384 2024-06-02T15:50:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.302+0000 7f1a5085c740 1 Processor -- start 2024-06-02T15:50:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.302+0000 7f1a5085c740 1 -- start start 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db24080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280125b0 con 0x56283db12cf0 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2800fe20 con 0x56283db136e0 2024-06-02T15:50:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a5085c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2800dc70 con 0x56283db18850 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a47400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a47400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:35178/0 (socket says 172.21.15.90:35178) 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.303+0000 7f1a47400640 1 -- 172.21.15.90:0/243086776 learned_addr learned my addr 172.21.15.90:0/243086776 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.304+0000 7f1a47400640 1 -- 172.21.15.90:0/243086776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 msgr2=0x56283db24080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.304+0000 7f1a46a00640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.304+0000 7f1a46000640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db24080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a47400640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db24080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a47400640 1 -- 172.21.15.90:0/243086776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 msgr2=0x56283db11be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a47400640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db11be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a47400640 1 -- 172.21.15.90:0/243086776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283db18560 con 0x56283db18850 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a46a00640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a46000640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db24080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.305+0000 7f1a47400640 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009b30 tx=0x7f1a4002ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f196a9e7ee95576 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.306+0000 7f1a33400640 1 -- 172.21.15.90:0/243086776 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40057040 con 0x56283db18850 2024-06-02T15:50:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.306+0000 7f1a33400640 1 -- 172.21.15.90:0/243086776 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x56283db18850 2024-06-02T15:50:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.306+0000 7f1a33400640 1 -- 172.21.15.90:0/243086776 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40036070 con 0x56283db18850 2024-06-02T15:50:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 -- 172.21.15.90:0/243086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 msgr2=0x56283db0dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009b30 tx=0x7f1a4002ae50 comp rx=0 tx=0).stop 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 -- 172.21.15.90:0/243086776 shutdown_connections 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db0dc40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db24080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 --2- 172.21.15.90:0/243086776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db11be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 -- 172.21.15.90:0/243086776 >> 172.21.15.90:0/243086776 conn(0x56283db0e360 msgr2=0x56283db14180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 -- 172.21.15.90:0/243086776 shutdown_connections 2024-06-02T15:50:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.307+0000 7f1a5085c740 1 -- 172.21.15.90:0/243086776 wait complete. 2024-06-02T15:50:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.308+0000 7f1a5085c740 1 -- start start 2024-06-02T15:50:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.308+0000 7f1a5085c740 1 Processor -- start 2024-06-02T15:50:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.309+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.309+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db21c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a46000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db21c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a46000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db21c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:36146/0 (socket says 172.21.15.90:36146) 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a5085c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a5085c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2800e2e0 con 0x56283db12cf0 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a5085c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2800f750 con 0x56283db136e0 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a5085c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2800e9b0 con 0x56283db18850 2024-06-02T15:50:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a46a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a5085c740 10 client.0 _ll_register_callbacks cb 0x7f1a40009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a46a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57436/0 (socket says 172.21.15.90:57436) 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.310+0000 7f1a46000640 1 -- 172.21.15.90:0/2223539628 learned_addr learned my addr 172.21.15.90:0/2223539628 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 -- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 msgr2=0x56283db21c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db21c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 -- 172.21.15.90:0/2223539628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 msgr2=0x56283db21720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a47400640 1 -- 172.21.15.90:0/2223539628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283db23260 con 0x56283db18850 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.311+0000 7f1a46a00640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.312+0000 7f1a47400640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db22c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56283db0c9c0 tx=0x7f1a4003e540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38511249c3ffb18 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.312+0000 7f1a30c00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.312+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40036070 con 0x56283db18850 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.312+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1a40053480 con 0x56283db18850 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.313+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40003970 con 0x56283db18850 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.313+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a40057040 con 0x56283db18850 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.314+0000 7f1a5085c740 10 client.25978 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.314+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56283db9b700 con 0x56283db18850 2024-06-02T15:50:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.314+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x56283db18850 2024-06-02T15:50:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1a40050070 con 0x56283db18850 2024-06-02T15:50:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 10 client.25978 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 20 client.25978 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db9bc80 0x56283db9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 4 client.25978 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:50:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.315+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56283da7e2c0 con 0x56283db9bc80 2024-06-02T15:50:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.316+0000 7f1a46a00640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db9bc80 0x56283db9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.316+0000 7f1a30c00640 10 client.25978 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:39.316+0000 7f1a46a00640 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db9bc80 0x56283db9e0a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d5ad5d877e50f2 server_cookie=5ba81e71862e45ec in_seq=0 out_seq=0 2024-06-02T15:50:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:39 smithi120 ceph-mon[36451]: pgmap v700: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 537 op/s 2024-06-02T15:50:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:39 smithi135 ceph-mon[36839]: pgmap v700: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 537 op/s 2024-06-02T15:50:39.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:39 smithi090 ceph-mon[31595]: pgmap v700: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 537 op/s 2024-06-02T15:50:39.878 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a30c00640 1 -- 172.21.15.90:0/2223539628 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56283da7e2c0 con 0x56283db9bc80 2024-06-02T15:50:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a30c00640 10 client.25978 handle_command_reply: tid=0 2024-06-02T15:50:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 client.25978 shutdown 2024-06-02T15:50:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 msgr2=0x56283db22c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db22c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56283db0c9c0 tx=0x7f1a4003e540 comp rx=0 tx=0).stop 2024-06-02T15:50:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 shutdown_connections 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x56283db9bc80 0x56283db9e0a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.413+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56283db18850 0x56283db22c70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.414+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x56283db136e0 0x56283db21c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.414+0000 7f1a5085c740 1 --2- 172.21.15.90:0/2223539628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56283db12cf0 0x56283db21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.414+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 >> 172.21.15.90:0/2223539628 conn(0x56283db0e360 msgr2=0x56283db14a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.414+0000 7f1a30c00640 0 client.25978 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.415+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 shutdown_connections 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.415+0000 7f1a5085c740 1 -- 172.21.15.90:0/2223539628 wait complete. 2024-06-02T15:50:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.415+0000 7f1a5085c740 20 client.25978 trim_cache size 0 max 16384 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 msgr2=0x7f1a2c0b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 0x7f1a2c0b4da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009560 tx=0x7f1a40049040 comp rx=0 tx=0).stop 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 msgr2=0x7f1a4810c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800d950 tx=0x7f1a3800de10 comp rx=0 tx=0).stop 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 shutdown_connections 2024-06-02T15:50:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1a2c0b28e0 0x7f1a2c0b4da0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a48110d90 0x7f1a4810b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1a4810a660 0x7f1a4810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 --2- 172.21.15.90:0/1631675783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48073fc0 0x7f1a4810c6e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.416+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 >> 172.21.15.90:0/1631675783 conn(0x7f1a4806f8f0 msgr2=0x7f1a4810cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.417+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 shutdown_connections 2024-06-02T15:50:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:40.417+0000 7f1a47e00640 1 -- 172.21.15.90:0/1631675783 wait complete. 2024-06-02T15:50:41.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:41 smithi090 ceph-mon[31595]: pgmap v701: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 311 op/s 2024-06-02T15:50:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:41 smithi120 ceph-mon[36451]: pgmap v701: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 311 op/s 2024-06-02T15:50:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:41 smithi135 ceph-mon[36839]: pgmap v701: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 311 op/s 2024-06-02T15:50:43.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:42 smithi120 ceph-mon[36451]: pgmap v702: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-02T15:50:43.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:42 smithi135 ceph-mon[36839]: pgmap v702: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-02T15:50:43.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:42 smithi090 ceph-mon[31595]: pgmap v702: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 349 op/s 2024-06-02T15:50:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:45 smithi120 ceph-mon[36451]: pgmap v703: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 398 op/s 2024-06-02T15:50:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:45 smithi135 ceph-mon[36839]: pgmap v703: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 398 op/s 2024-06-02T15:50:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:45 smithi090 ceph-mon[31595]: pgmap v703: 129 pgs: 129 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 398 op/s 2024-06-02T15:50:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:47 smithi120 ceph-mon[36451]: pgmap v704: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 571 op/s 2024-06-02T15:50:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:47 smithi135 ceph-mon[36839]: pgmap v704: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 571 op/s 2024-06-02T15:50:47.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:47 smithi090 ceph-mon[31595]: pgmap v704: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 571 op/s 2024-06-02T15:50:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:49 smithi120 ceph-mon[36451]: pgmap v705: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 555 op/s 2024-06-02T15:50:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:49 smithi135 ceph-mon[36839]: pgmap v705: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 555 op/s 2024-06-02T15:50:49.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:49 smithi090 ceph-mon[31595]: pgmap v705: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 555 op/s 2024-06-02T15:50:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:51 smithi120 ceph-mon[36451]: pgmap v706: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 551 op/s 2024-06-02T15:50:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:51 smithi135 ceph-mon[36839]: pgmap v706: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 551 op/s 2024-06-02T15:50:51.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:51 smithi090 ceph-mon[31595]: pgmap v706: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 551 op/s 2024-06-02T15:50:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:53 smithi120 ceph-mon[36451]: pgmap v707: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 539 op/s 2024-06-02T15:50:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:53 smithi135 ceph-mon[36839]: pgmap v707: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 539 op/s 2024-06-02T15:50:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 -- 172.21.15.90:0/3376963416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 msgr2=0x7f1d6010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 --2- 172.21.15.90:0/3376963416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d6010af10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ca0 tx=0x7f1d5802a8c0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 -- 172.21.15.90:0/3376963416 shutdown_connections 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 --2- 172.21.15.90:0/3376963416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d6010af10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 --2- 172.21.15.90:0/3376963416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60103cc0 0x7f1d60104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 --2- 172.21.15.90:0/3376963416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60102ad0 0x7f1d60102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.577+0000 7f1d65e00640 1 -- 172.21.15.90:0/3376963416 >> 172.21.15.90:0/3376963416 conn(0x7f1d600fe040 msgr2=0x7f1d60100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.578+0000 7f1d65e00640 1 -- 172.21.15.90:0/3376963416 shutdown_connections 2024-06-02T15:50:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.578+0000 7f1d65e00640 1 -- 172.21.15.90:0/3376963416 wait complete. 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 Processor -- start 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 -- start start 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60103cc0 0x7f1d60074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d60075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010e650 con 0x7f1d60102ad0 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010e4d0 con 0x7f1d60104680 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d65e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6010e7d0 con 0x7f1d60103cc0 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40390/0 (socket says 172.21.15.90:40390) 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d5f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d60075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.579+0000 7f1d5ea00640 1 -- 172.21.15.90:0/4123718706 learned_addr learned my addr 172.21.15.90:0/4123718706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5e000640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60103cc0 0x7f1d60074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 -- 172.21.15.90:0/4123718706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60103cc0 msgr2=0x7f1d60074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60103cc0 0x7f1d60074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 -- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 msgr2=0x7f1d60076800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 -- 172.21.15.90:0/4123718706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60075c10 con 0x7f1d60104680 2024-06-02T15:50:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.580+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d60075380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d58031a20 tx=0x7f1d580316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.581+0000 7f1d64a00640 1 -- 172.21.15.90:0/4123718706 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58003b70 con 0x7f1d60104680 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.581+0000 7f1d64a00640 1 -- 172.21.15.90:0/4123718706 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58003d10 con 0x7f1d60104680 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.581+0000 7f1d64a00640 1 -- 172.21.15.90:0/4123718706 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5802fae0 con 0x7f1d60104680 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.581+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601a2d60 con 0x7f1d60104680 2024-06-02T15:50:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.581+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a31a0 con 0x7f1d60104680 2024-06-02T15:50:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.584+0000 7f1d68b86740 1 Processor -- start 2024-06-02T15:50:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.584+0000 7f1d64a00640 1 -- 172.21.15.90:0/4123718706 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1d58004610 con 0x7f1d60104680 2024-06-02T15:50:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 -- start start 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f35b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556b57e9db90 con 0x556b57f33690 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b57e8d730 con 0x556b57f2e520 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d68b86740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b57e91770 con 0x556b57f2db30 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42090/0 (socket says 172.21.15.90:42090) 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d5ea00640 1 -- 172.21.15.90:0/1629439336 learned_addr learned my addr 172.21.15.90:0/1629439336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d5e000640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.585+0000 7f1d5e000640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f35b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 -- 172.21.15.90:0/1629439336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 msgr2=0x556b57f2df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 -- 172.21.15.90:0/1629439336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 msgr2=0x556b57f35b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f35b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 -- 172.21.15.90:0/1629439336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b57e268b0 con 0x556b57f2e520 2024-06-02T15:50:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5e000640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f35b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d500115f0 tx=0x7f1d50011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6461d947174842 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d64a00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 0x7f1d440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d4be00640 1 -- 172.21.15.90:0/1629439336 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50004020 con 0x556b57f2e520 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d4be00640 1 -- 172.21.15.90:0/1629439336 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d500041c0 con 0x556b57f2e520 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d4be00640 1 -- 172.21.15.90:0/1629439336 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500038f0 con 0x556b57f2e520 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.586+0000 7f1d64a00640 1 -- 172.21.15.90:0/4123718706 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1d581038d0 con 0x7f1d60104680 2024-06-02T15:50:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d5e000640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 0x7f1d440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 -- 172.21.15.90:0/1629439336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 msgr2=0x556b57f33020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d500115f0 tx=0x7f1d50011ab0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 -- 172.21.15.90:0/1629439336 shutdown_connections 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f35b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f33020 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 --2- 172.21.15.90:0/1629439336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 -- 172.21.15.90:0/1629439336 >> 172.21.15.90:0/1629439336 conn(0x556b57f291a0 msgr2=0x556b57f295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d5e000640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 0x7f1d440b4de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x556b57f2ed80 tx=0x7f1d4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 -- 172.21.15.90:0/1629439336 shutdown_connections 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.587+0000 7f1d68b86740 1 -- 172.21.15.90:0/1629439336 wait complete. 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.588+0000 7f1d68b86740 1 -- start start 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.588+0000 7f1d68b86740 1 Processor -- start 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f37950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f37e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556b57b73790 con 0x556b57f2e520 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556b57e35bc0 con 0x556b57f33690 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b57f36640 con 0x556b57f2db30 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d68b86740 10 client.0 _ll_register_callbacks cb 0x556b57e13400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f37e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f37950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42094/0 (socket says 172.21.15.90:42094) 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 -- 172.21.15.90:0/4218367336 learned_addr learned my addr 172.21.15.90:0/4218367336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 -- 172.21.15.90:0/4218367336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 msgr2=0x556b57f37950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f37950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 -- 172.21.15.90:0/4218367336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 msgr2=0x556b57f37e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.589+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f37e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d5f400640 1 -- 172.21.15.90:0/4218367336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b57ba2210 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d5f400640 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d580caa70 tx=0x7f1d580ca310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e124ae5dd9d7f89a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d48c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580cb1e0 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1d5802fc80 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d500041c0 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d580caae0 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d68b86740 10 client.26057 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.590+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b57c6e620 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d500041c0 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1d580cad40 con 0x556b57f33690 2024-06-02T15:50:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 10 client.26057 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 10 client.26057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 20 client.26057 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57f43380 0x556b57f45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 4 client.26057 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b57e9fae0 con 0x556b57f43380 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.591+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57f43380 0x556b57f45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.592+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57f43380 0x556b57f45810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f507f51466ba22e7 server_cookie=8d7cc42dbe765a53 in_seq=0 out_seq=0 2024-06-02T15:50:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.592+0000 7f1d48c00640 10 client.26057 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d48c00640 1 -- 172.21.15.90:0/4218367336 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b57e9fae0 con 0x556b57f43380 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d48c00640 10 client.26057 handle_command_reply: tid=0 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 client.26057 shutdown 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 msgr2=0x556b57f3eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d580caa70 tx=0x7f1d580ca310 comp rx=0 tx=0).stop 2024-06-02T15:50:53.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 shutdown_connections 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57f43380 0x556b57f45810 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f3eed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f37e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d48c00640 0 client.26057 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 --2- 172.21.15.90:0/4218367336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f37950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 >> 172.21.15.90:0/4218367336 conn(0x556b57f291a0 msgr2=0x556b57f28310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 shutdown_connections 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 1 -- 172.21.15.90:0/4218367336 wait complete. 2024-06-02T15:50:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.593+0000 7f1d68b86740 20 client.26057 trim_cache size 0 max 16384 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 Processor -- start 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 -- start start 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f2f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601b0060 con 0x556b57f2e520 2024-06-02T15:50:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ad8d0 con 0x556b57f33690 2024-06-02T15:50:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.612+0000 7f1d68b86740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ab720 con 0x556b57f2db30 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40418/0 (socket says 172.21.15.90:40418) 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 -- 172.21.15.90:0/3744407139 learned_addr learned my addr 172.21.15.90:0/3744407139 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5f400640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f2f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 -- 172.21.15.90:0/3744407139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 msgr2=0x556b57f2c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 -- 172.21.15.90:0/3744407139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 msgr2=0x556b57f2f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f2f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5e000640 1 -- 172.21.15.90:0/3744407139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b57f276b0 con 0x556b57f2e520 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.613+0000 7f1d5f400640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f2f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.614+0000 7f1d5e000640 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c01d730 tx=0x7f1d4c013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f4cf6e46e9ddb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.614+0000 7f1d4b400640 1 -- 172.21.15.90:0/3744407139 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c029460 con 0x556b57f2e520 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.614+0000 7f1d4b400640 1 -- 172.21.15.90:0/3744407139 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4c029ab0 con 0x556b57f2e520 2024-06-02T15:50:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.614+0000 7f1d4b400640 1 -- 172.21.15.90:0/3744407139 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c01c910 con 0x556b57f2e520 2024-06-02T15:50:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- 172.21.15.90:0/3744407139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 msgr2=0x556b57f3ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c01d730 tx=0x7f1d4c013050 comp rx=0 tx=0).stop 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- 172.21.15.90:0/3744407139 shutdown_connections 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f33690 0x556b57f2f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f2e520 0x556b57f3ee90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 --2- 172.21.15.90:0/3744407139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f2c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- 172.21.15.90:0/3744407139 >> 172.21.15.90:0/3744407139 conn(0x556b57f291a0 msgr2=0x556b57f2efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- 172.21.15.90:0/3744407139 shutdown_connections 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- 172.21.15.90:0/3744407139 wait complete. 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.615+0000 7f1d68b86740 1 -- start start 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 Processor -- start 2024-06-02T15:50:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601abd90 con 0x556b57f33690 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ad200 con 0x556b57f2e520 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ac460 con 0x556b57f2db30 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d68b86740 10 client.0 _ll_register_callbacks cb 0x7f1d58005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d5e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40424/0 (socket says 172.21.15.90:40424) 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d5e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42106/0 (socket says 172.21.15.90:42106) 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.616+0000 7f1d5f400640 1 -- 172.21.15.90:0/246967194 learned_addr learned my addr 172.21.15.90:0/246967194 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 -- 172.21.15.90:0/246967194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 msgr2=0x556b57f3c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f3c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 -- 172.21.15.90:0/246967194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 msgr2=0x556b57f3dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 -- 172.21.15.90:0/246967194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b57f3e210 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5f400640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d5e000640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x556b57f38860 tx=0x7f1d4c029690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62f85e91b464c21 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d48c00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c005040 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.617+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1d4c0051e0 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d4c029ab0 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d4c03f050 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 10 client.26063 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556b57fb6980 con 0x556b57f2e520 2024-06-02T15:50:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4c029ab0 con 0x556b57f2e520 2024-06-02T15:50:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1d4c03a030 con 0x556b57f2e520 2024-06-02T15:50:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 10 client.26063 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 20 client.26063 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57fb7350 0x556b57fb9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 4 client.26063 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.618+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b57e992c0 con 0x556b57fb7350 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.619+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57fb7350 0x556b57fb9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.619+0000 7f1d5ea00640 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57fb7350 0x556b57fb9770 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2135cc1020f9d1fd server_cookie=8c82d41a105cf930 in_seq=0 out_seq=0 2024-06-02T15:50:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:53.619+0000 7f1d48c00640 10 client.26063 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:53 smithi090 ceph-mon[31595]: pgmap v707: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 539 op/s 2024-06-02T15:50:54.860 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:50:54.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.208+0000 7f7d02400700 1 -- 172.21.15.90:0/1937523863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc108470 msgr2=0x7f7cfc06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.208+0000 7f7d02400700 1 --2- 172.21.15.90:0/1937523863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc108470 0x7f7cfc06b250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0098f0 tx=0x7f7cec009c00 comp rx=0 tx=0).stop 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.210+0000 7f7d02400700 1 -- 172.21.15.90:0/1937523863 shutdown_connections 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.210+0000 7f7d02400700 1 --2- 172.21.15.90:0/1937523863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cfc105f80 0x7f7cfc106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.210+0000 7f7d02400700 1 --2- 172.21.15.90:0/1937523863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc108470 0x7f7cfc06b250 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.210+0000 7f7d02400700 1 --2- 172.21.15.90:0/1937523863 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc06b790 0x7f7cfc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.210+0000 7f7d02400700 1 -- 172.21.15.90:0/1937523863 >> 172.21.15.90:0/1937523863 conn(0x7f7cfc0793f0 msgr2=0x7f7cfc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.212+0000 7f7d02400700 1 -- 172.21.15.90:0/1937523863 shutdown_connections 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.213+0000 7f7d02400700 1 -- 172.21.15.90:0/1937523863 wait complete. 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.216+0000 7f7d02400700 1 Processor -- start 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.217+0000 7f7d02400700 1 -- start start 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.217+0000 7f7d02400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 0x7f7cfc1a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.217+0000 7f7d02400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cfc105f80 0x7f7cfc1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.218+0000 7f7d02400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 0x7f7cfc1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.218+0000 7f7d02400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc119d00 con 0x7f7cfc105f80 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.218+0000 7f7d02400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc119b80 con 0x7f7cfc108470 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.218+0000 7f7d02400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc119e80 con 0x7f7cfc06b790 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.220+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 0x7f7cfc1a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.220+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 0x7f7cfc1a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52112/0 (socket says 172.21.15.90:52112) 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.220+0000 7f7cfb400700 1 -- 172.21.15.90:0/2255340212 learned_addr learned my addr 172.21.15.90:0/2255340212 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.220+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 0x7f7cfc1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 -- 172.21.15.90:0/2255340212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 msgr2=0x7f7cfc1a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 0x7f7cfc1a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 -- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cfc105f80 msgr2=0x7f7cfc1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cfc105f80 0x7f7cfc1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 -- 172.21.15.90:0/2255340212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc1a52e0 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 0x7f7cfc1a4ba0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009cd0 tx=0x7f7cf400c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.221+0000 7f7cf3e00700 1 -- 172.21.15.90:0/2255340212 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40090d0 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.223+0000 7f7cf3e00700 1 -- 172.21.15.90:0/2255340212 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf4009270 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.223+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc1a5570 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.223+0000 7f7cf3e00700 1 -- 172.21.15.90:0/2255340212 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40046c0 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.223+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc1ace50 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.232+0000 7f7cf3e00700 1 -- 172.21.15.90:0/2255340212 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7cf401a020 con 0x7f7cfc108470 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.232+0000 7f7d0ce00b80 1 Processor -- start 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.234+0000 7f7cf3e00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 0x7f7ce00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 -- start start 2024-06-02T15:50:54.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7cf3e00700 1 -- 172.21.15.90:0/2255340212 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7cf40e1f30 con 0x7f7cfc108470 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 0x7f7ce00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559522e7ee40 con 0x559522f21140 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559522e6e9e0 con 0x559522f26f30 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.235+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559522e72a20 con 0x559522f26610 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.236+0000 7f7cfaa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.236+0000 7f7cfaa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42040/0 (socket says 172.21.15.90:42040) 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.236+0000 7f7cfaa00700 1 -- 172.21.15.90:0/3228833151 learned_addr learned my addr 172.21.15.90:0/3228833151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.236+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.237+0000 7f7cfaa00700 1 -- 172.21.15.90:0/3228833151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 msgr2=0x559522f269f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.237+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f269f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.237+0000 7f7cfaa00700 1 -- 172.21.15.90:0/3228833151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 msgr2=0x559522f215c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.237+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f215c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.237+0000 7f7cfaa00700 1 -- 172.21.15.90:0/3228833151 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559522e37420 con 0x559522f26f30 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.238+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 0x7f7ce00b4e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8012850 tx=0x7f7ce80123f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.238+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8012fd0 tx=0x7f7ce801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bc670c3b3115336 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.238+0000 7f7cf0c00700 1 -- 172.21.15.90:0/3228833151 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8004830 con 0x559522f26f30 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.238+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.239+0000 7f7cf0c00700 1 -- 172.21.15.90:0/3228833151 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce8005020 con 0x559522f26f30 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.239+0000 7f7cf0c00700 1 -- 172.21.15.90:0/3228833151 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce801f650 con 0x559522f26f30 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3228833151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 msgr2=0x559522f20b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8012fd0 tx=0x7f7ce801a000 comp rx=0 tx=0).stop 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3228833151 shutdown_connections 2024-06-02T15:50:54.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3228833151 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f20b20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3228833151 >> 172.21.15.90:0/3228833151 conn(0x559522e4ebe0 msgr2=0x559522f1b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3228833151 shutdown_connections 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.240+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3228833151 wait complete. 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.241+0000 7f7d0ce00b80 1 -- start start 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 Processor -- start 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26f30 0x559522f2a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559522d74fa0 con 0x559522f26f30 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559522e0f760 con 0x559522f21140 2024-06-02T15:50:54.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.242+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559522f3e9e0 con 0x559522f26610 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.243+0000 7f7d0ce00b80 10 client.0 _ll_register_callbacks cb 0x559522e18c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.243+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.243+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42046/0 (socket says 172.21.15.90:42046) 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.243+0000 7f7cfb400700 1 -- 172.21.15.90:0/4148059816 learned_addr learned my addr 172.21.15.90:0/4148059816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26f30 0x559522f2a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfb400700 1 -- 172.21.15.90:0/4148059816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 msgr2=0x559522f2a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfb400700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f2a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfb400700 1 -- 172.21.15.90:0/4148059816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26f30 msgr2=0x559522f2a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfb400700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26f30 0x559522f2a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.244+0000 7f7cfb400700 1 -- 172.21.15.90:0/4148059816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559522dbe8d0 con 0x559522f21140 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.245+0000 7f7cfb400700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00f690 tx=0x7f7cec009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f201dd2eea4149 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.245+0000 7f7ce5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.245+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec005450 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.245+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7cec0055f0 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.246+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce8005020 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.246+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7cec00f9d0 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.246+0000 7f7d0ce00b80 10 client.26039 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.246+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559522e3f060 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.247+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce8005020 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.247+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f7cec02b030 con 0x559522f21140 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.247+0000 7f7d0ce00b80 10 client.26039 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.248+0000 7f7d0ce00b80 10 client.26039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.248+0000 7f7d0ce00b80 20 client.26039 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:54.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.248+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522f4b5e0 0x559522f4d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.248+0000 7f7d0ce00b80 4 client.26039 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.248+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559522e80d90 con 0x559522f4b5e0 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.249+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522f4b5e0 0x559522f4d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.250+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522f4b5e0 0x559522f4d9d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7450a7e7653dbef9 server_cookie=7fc986db90455aeb in_seq=0 out_seq=0 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.250+0000 7f7ce5600700 10 client.26039 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7ce5600700 1 -- 172.21.15.90:0/4148059816 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559522e80d90 con 0x559522f4b5e0 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7ce5600700 10 client.26039 handle_command_reply: tid=0 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 client.26039 shutdown 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 msgr2=0x559522f2cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00f690 tx=0x7f7cec009640 comp rx=0 tx=0).stop 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 shutdown_connections 2024-06-02T15:50:54.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522f4b5e0 0x559522f4d9d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26f30 0x559522f2a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f2cc60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/4148059816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f2a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 >> 172.21.15.90:0/4148059816 conn(0x559522e4ebe0 msgr2=0x559522f1ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7ce5600700 0 client.26039 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.252+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 shutdown_connections 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.253+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/4148059816 wait complete. 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.253+0000 7f7d0ce00b80 20 client.26039 trim_cache size 0 max 16384 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.283+0000 7f7d0ce00b80 1 Processor -- start 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.283+0000 7f7d0ce00b80 1 -- start start 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc1afa70 con 0x559522f21140 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc1ad2a0 con 0x559522f26f30 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc073840 con 0x559522f26610 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.284+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40358/0 (socket says 172.21.15.90:40358) 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42054/0 (socket says 172.21.15.90:42054) 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52122/0 (socket says 172.21.15.90:52122) 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 -- 172.21.15.90:0/3251111022 learned_addr learned my addr 172.21.15.90:0/3251111022 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 -- 172.21.15.90:0/3251111022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 msgr2=0x559522f532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 -- 172.21.15.90:0/3251111022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 msgr2=0x559522f1b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.285+0000 7f7cfbe00700 1 -- 172.21.15.90:0/3251111022 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559522f28590 con 0x559522f26f30 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.286+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.286+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40a63b0 tx=0x7f7cf40a5080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624c53fdf90cee7f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.286+0000 7f7ce7e00700 1 -- 172.21.15.90:0/3251111022 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40a88b0 con 0x559522f26f30 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.286+0000 7f7ce7e00700 1 -- 172.21.15.90:0/3251111022 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf40a8c50 con 0x559522f26f30 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.286+0000 7f7ce7e00700 1 -- 172.21.15.90:0/3251111022 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40b1fa0 con 0x559522f26f30 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.287+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3251111022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 msgr2=0x559522f53820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.287+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40a63b0 tx=0x7f7cf40a5080 comp rx=0 tx=0).stop 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3251111022 shutdown_connections 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f26f30 0x559522f53820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f21140 0x559522f1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3251111022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26610 0x559522f532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3251111022 >> 172.21.15.90:0/3251111022 conn(0x559522e4ebe0 msgr2=0x559522f28910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3251111022 shutdown_connections 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.288+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3251111022 wait complete. 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.289+0000 7f7d0ce00b80 1 -- start start 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.289+0000 7f7d0ce00b80 1 Processor -- start 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26610 0x559522f53330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 0x559522f53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc073eb0 con 0x559522f26610 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc075320 con 0x559522f21140 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc074580 con 0x559522f26f30 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.290+0000 7f7d0ce00b80 10 client.0 _ll_register_callbacks cb 0x559522e18c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.291+0000 7f7cfaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26610 0x559522f53330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.291+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.291+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42070/0 (socket says 172.21.15.90:42070) 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.291+0000 7f7cfb400700 1 -- 172.21.15.90:0/3252558737 learned_addr learned my addr 172.21.15.90:0/3252558737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.291+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 0x559522f53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 -- 172.21.15.90:0/3252558737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 msgr2=0x559522f53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 0x559522f53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 -- 172.21.15.90:0/3252558737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26610 msgr2=0x559522f53330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26610 0x559522f53330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 -- 172.21.15.90:0/3252558737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559522fb5200 con 0x559522f21140 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfbe00700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 0x559522f53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.292+0000 7f7cfb400700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0098c0 tx=0x7f7cec00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab85442ec66f104 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.293+0000 7f7ce5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.293+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec005660 con 0x559522f21140 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.293+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf40a8c50 con 0x559522f21140 2024-06-02T15:50:54.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.293+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7cec0046d0 con 0x559522f21140 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.294+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7cec042050 con 0x559522f21140 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.295+0000 7f7d0ce00b80 10 client.26045 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.295+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559522fb64c0 con 0x559522f21140 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.295+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf40a8c50 con 0x559522f21140 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f7cec03d030 con 0x559522f21140 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 10 client.26045 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 10 client.26045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 20 client.26045 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522fb6c50 0x559522fb9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 4 client.26045 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.296+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559522e7a570 con 0x559522fb6c50 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.297+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522fb6c50 0x559522fb9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.297+0000 7f7cfaa00700 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522fb6c50 0x559522fb9070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef4522118a75400 server_cookie=f20cd469086aebc5 in_seq=0 out_seq=0 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.297+0000 7f7ce5600700 10 client.26045 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:54.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.297+0000 7f7ce5600700 1 -- 172.21.15.90:0/3252558737 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559522e7a570 con 0x559522fb6c50 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7ce5600700 10 client.26045 handle_command_reply: tid=0 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 client.26045 shutdown 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 msgr2=0x559522f52df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0098c0 tx=0x7f7cec00b810 comp rx=0 tx=0).stop 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 shutdown_connections 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559522fb6c50 0x559522fb9070 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559522f26f30 0x559522f53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559522f21140 0x559522f52df0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 --2- 172.21.15.90:0/3252558737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559522f26610 0x559522f53330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 >> 172.21.15.90:0/3252558737 conn(0x559522e4ebe0 msgr2=0x559522f27310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7ce5600700 0 client.26045 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:54.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 shutdown_connections 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.298+0000 7f7d0ce00b80 1 -- 172.21.15.90:0/3252558737 wait complete. 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.299+0000 7f7d0ce00b80 20 client.26045 trim_cache size 0 max 16384 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.299+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 msgr2=0x7f7ce00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.299+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 0x7f7ce00b4e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8012850 tx=0x7f7ce80123f0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.299+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 msgr2=0x7f7cfc1a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.299+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 0x7f7cfc1a4ba0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009cd0 tx=0x7f7cf400c6d0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 shutdown_connections 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7ce00b2980 0x7f7ce00b4e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7cfc108470 0x7f7cfc1a4ba0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cfc06b790 0x7f7cfc1a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 --2- 172.21.15.90:0/2255340212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7cfc105f80 0x7f7cfc1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 >> 172.21.15.90:0/2255340212 conn(0x7f7cfc0793f0 msgr2=0x7f7cfc104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:54.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 shutdown_connections 2024-06-02T15:50:54.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:52.300+0000 7f7d02400700 1 -- 172.21.15.90:0/2255340212 wait complete. 2024-06-02T15:50:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d48c00640 1 -- 172.21.15.90:0/246967194 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b57e992c0 con 0x556b57fb7350 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d48c00640 10 client.26063 handle_command_reply: tid=0 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 client.26063 shutdown 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 msgr2=0x556b57f3cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x556b57f38860 tx=0x7f1d4c029690 comp rx=0 tx=0).stop 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 shutdown_connections 2024-06-02T15:50:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556b57fb7350 0x556b57fb9770 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b57f33690 0x556b57f3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556b57f2e520 0x556b57f3cc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 --2- 172.21.15.90:0/246967194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b57f2db30 0x556b57f3c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 >> 172.21.15.90:0/246967194 conn(0x556b57f291a0 msgr2=0x556b57f2f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d48c00640 0 client.26063 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.153+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 shutdown_connections 2024-06-02T15:50:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d68b86740 1 -- 172.21.15.90:0/246967194 wait complete. 2024-06-02T15:50:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d68b86740 20 client.26063 trim_cache size 0 max 16384 2024-06-02T15:50:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:50:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:50:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:50:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 msgr2=0x7f1d440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 0x7f1d440b4de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x556b57f2ed80 tx=0x7f1d4c005f90 comp rx=0 tx=0).stop 2024-06-02T15:50:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 msgr2=0x7f1d60075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.154+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d60075380 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d58031a20 tx=0x7f1d580316f0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 shutdown_connections 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1d440b2920 0x7f1d440b4de0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d60104680 0x7f1d60075380 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d60103cc0 0x7f1d60074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 --2- 172.21.15.90:0/4123718706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d60102ad0 0x7f1d60076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 >> 172.21.15.90:0/4123718706 conn(0x7f1d600fe040 msgr2=0x7f1d60106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 shutdown_connections 2024-06-02T15:50:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:50:55.155+0000 7f1d65e00640 1 -- 172.21.15.90:0/4123718706 wait complete. 2024-06-02T15:50:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:55 smithi120 ceph-mon[36451]: pgmap v708: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-06-02T15:50:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:55 smithi135 ceph-mon[36839]: pgmap v708: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-06-02T15:50:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:55 smithi090 ceph-mon[31595]: pgmap v708: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 1.1 MiB/s wr, 256 op/s 2024-06-02T15:50:56.675 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:50:56.675 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:50:56.968 INFO:tasks.workunit.client.0.smithi090.stdout:created hierarchy, now cleaning up 2024-06-02T15:50:57.165 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:50:57.557 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:57 smithi090 ceph-mon[31595]: pgmap v709: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-06-02T15:50:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:57 smithi120 ceph-mon[36451]: pgmap v709: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-06-02T15:50:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:57 smithi135 ceph-mon[36839]: pgmap v709: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 390 op/s 2024-06-02T15:50:59.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:50:59 smithi090 ceph-mon[31595]: pgmap v710: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.0 MiB/s wr, 332 op/s 2024-06-02T15:50:59.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.550+0000 7f72f3200700 1 -- 172.21.15.90:0/2158179602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec111d20 msgr2=0x7f72ec10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.550+0000 7f72f3200700 1 --2- 172.21.15.90:0/2158179602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec111d20 0x7f72ec10a540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009990 tx=0x7f72dc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 -- 172.21.15.90:0/2158179602 shutdown_connections 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 --2- 172.21.15.90:0/2158179602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec11cd20 0x7f72ec11f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 --2- 172.21.15.90:0/2158179602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec111d20 0x7f72ec10a540 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72dc009990 tx=0x7f72dc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 --2- 172.21.15.90:0/2158179602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec10aa80 0x7f72ec10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 -- 172.21.15.90:0/2158179602 >> 172.21.15.90:0/2158179602 conn(0x7f72ec06e6b0 msgr2=0x7f72ec06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 -- 172.21.15.90:0/2158179602 shutdown_connections 2024-06-02T15:50:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.551+0000 7f72f3200700 1 -- 172.21.15.90:0/2158179602 wait complete. 2024-06-02T15:50:59.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.553+0000 7f72f3200700 1 Processor -- start 2024-06-02T15:50:59.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.555+0000 7f72f3200700 1 -- start start 2024-06-02T15:50:59.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 0x7f72ec11b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec11cd20 0x7f72ec114690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec122660 con 0x7f72ec10aa80 2024-06-02T15:50:59.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1224e0 con 0x7f72ec114bd0 2024-06-02T15:50:59.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.556+0000 7f72f3200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1227e0 con 0x7f72ec11cd20 2024-06-02T15:50:59.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42130/0 (socket says 172.21.15.90:42130) 2024-06-02T15:50:59.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 -- 172.21.15.90:0/4226044570 learned_addr learned my addr 172.21.15.90:0/4226044570 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:59.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 -- 172.21.15.90:0/4226044570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec11cd20 msgr2=0x7f72ec114690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec11cd20 0x7f72ec114690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 -- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 msgr2=0x7f72ec11b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72ebe00700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 0x7f72ec11b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 0x7f72ec11b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.557+0000 7f72f1200700 1 -- 172.21.15.90:0/4226044570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec119320 con 0x7f72ec114bd0 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.558+0000 7f72f1200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72e4007ec0 tx=0x7f72e400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.558+0000 7f72e8c00700 1 -- 172.21.15.90:0/4226044570 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4004bb0 con 0x7f72ec114bd0 2024-06-02T15:50:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:50:59 smithi120 ceph-mon[36451]: pgmap v710: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.0 MiB/s wr, 332 op/s 2024-06-02T15:50:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.558+0000 7f72e8c00700 1 -- 172.21.15.90:0/4226044570 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4004d50 con 0x7f72ec114bd0 2024-06-02T15:50:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.558+0000 7f72e8c00700 1 -- 172.21.15.90:0/4226044570 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4004550 con 0x7f72ec114bd0 2024-06-02T15:50:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.558+0000 7f72ebe00700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 0x7f72ec11b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.560+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec119610 con 0x7f72ec114bd0 2024-06-02T15:50:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.560+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec119ad0 con 0x7f72ec114bd0 2024-06-02T15:50:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.566+0000 7f72e8c00700 1 -- 172.21.15.90:0/4226044570 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f72e40046f0 con 0x7f72ec114bd0 2024-06-02T15:50:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 Processor -- start 2024-06-02T15:50:59.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 -- start start 2024-06-02T15:50:59.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5650fbbdbec0 con 0x5650fbc7e1c0 2024-06-02T15:50:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:50:59 smithi135 ceph-mon[36839]: pgmap v710: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.0 MiB/s wr, 332 op/s 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650fbbcba60 con 0x5650fbc83690 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.567+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650fbbcfaa0 con 0x5650fbc83fb0 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.568+0000 7f72eb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.568+0000 7f72e8c00700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 0x7f72d00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.568+0000 7f72eb400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42138/0 (socket says 172.21.15.90:42138) 2024-06-02T15:50:59.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.568+0000 7f72eb400700 1 -- 172.21.15.90:0/4026127131 learned_addr learned my addr 172.21.15.90:0/4026127131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.568+0000 7f72f1200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 0x7f72d00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 -- 172.21.15.90:0/4026127131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 msgr2=0x5650fbc7dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 -- 172.21.15.90:0/4026127131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 msgr2=0x5650fbc7e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 -- 172.21.15.90:0/4026127131 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fbb944a0 con 0x5650fbc83690 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72eb400700 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f72d8001760 tx=0x7f72d8001a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab247bcca3e82b5a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72e1600700 1 -- 172.21.15.90:0/4026127131 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004c70 con 0x5650fbc83690 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72e1600700 1 -- 172.21.15.90:0/4026127131 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004e10 con 0x5650fbc83690 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72e1600700 1 -- 172.21.15.90:0/4026127131 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8009280 con 0x5650fbc83690 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.569+0000 7f72e8c00700 1 -- 172.21.15.90:0/4226044570 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f72e40e1840 con 0x7f72ec114bd0 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72f1200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 0x7f72d00b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72e40def00 tx=0x7f72e40e7240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4026127131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 msgr2=0x5650fbc83a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f72d8001760 tx=0x7f72d8001a70 comp rx=0 tx=0).stop 2024-06-02T15:50:59.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4026127131 shutdown_connections 2024-06-02T15:50:59.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc83a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4026127131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc7dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.570+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4026127131 >> 172.21.15.90:0/4026127131 conn(0x5650fbbabc60 msgr2=0x5650fbc78710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.571+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4026127131 shutdown_connections 2024-06-02T15:50:59.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.572+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4026127131 wait complete. 2024-06-02T15:50:59.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.572+0000 7f72fdbcab80 1 -- start start 2024-06-02T15:50:59.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.572+0000 7f72fdbcab80 1 Processor -- start 2024-06-02T15:50:59.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc89db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5650fbad2020 con 0x5650fbc7e1c0 2024-06-02T15:50:59.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5650fbb6c7e0 con 0x5650fbc83690 2024-06-02T15:50:59.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650fbc8aa60 con 0x5650fbc83fb0 2024-06-02T15:50:59.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.574+0000 7f72fdbcab80 10 client.0 _ll_register_callbacks cb 0x5650fbb75cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc89db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42142/0 (socket says 172.21.15.90:42142) 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 -- 172.21.15.90:0/2088593578 learned_addr learned my addr 172.21.15.90:0/2088593578 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 -- 172.21.15.90:0/2088593578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 msgr2=0x5650fbc87610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.575+0000 7f72ebe00700 1 -- 172.21.15.90:0/2088593578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 msgr2=0x5650fbc89db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.576+0000 7f72ebe00700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc89db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.576+0000 7f72ebe00700 1 -- 172.21.15.90:0/2088593578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fbb1b950 con 0x5650fbc83690 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.576+0000 7f72eb400700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc89db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.576+0000 7f72ebe00700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72dc006010 tx=0x7f72dc00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=139166440f5cb161 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72d6000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc026060 con 0x5650fbc83690 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d8004e10 con 0x5650fbc83690 2024-06-02T15:50:59.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f72dc003a60 con 0x5650fbc83690 2024-06-02T15:50:59.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72dc02d050 con 0x5650fbc83690 2024-06-02T15:50:59.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72fdbcab80 10 client.26075 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:59.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.577+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5650fbb9c0e0 con 0x5650fbc83690 2024-06-02T15:50:59.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.578+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004e10 con 0x5650fbc83690 2024-06-02T15:50:59.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f72dc00f460 con 0x5650fbc83690 2024-06-02T15:50:59.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 10 client.26075 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 20 client.26075 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbc97720 0x5650fbc99b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 4 client.26075 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.579+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650fbbdde10 con 0x5650fbc97720 2024-06-02T15:50:59.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.580+0000 7f72eb400700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbc97720 0x5650fbc99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.581+0000 7f72eb400700 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbc97720 0x5650fbc99b80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8797bdd9d8070df server_cookie=9cabdedcbcbbef22 in_seq=0 out_seq=0 2024-06-02T15:50:59.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.581+0000 7f72d6000700 10 client.26075 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.582+0000 7f72d6000700 1 -- 172.21.15.90:0/2088593578 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5650fbbdde10 con 0x5650fbc97720 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.582+0000 7f72d6000700 10 client.26075 handle_command_reply: tid=0 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 client.26075 shutdown 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 msgr2=0x5650fbc870d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72dc006010 tx=0x7f72dc00bf90 comp rx=0 tx=0).stop 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 shutdown_connections 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbc97720 0x5650fbc99b80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc89db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/2088593578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83690 0x5650fbc870d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 >> 172.21.15.90:0/2088593578 conn(0x5650fbbabc60 msgr2=0x5650fbc77df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72d6000700 0 client.26075 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:59.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 shutdown_connections 2024-06-02T15:50:59.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.583+0000 7f72fdbcab80 1 -- 172.21.15.90:0/2088593578 wait complete. 2024-06-02T15:50:59.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.584+0000 7f72fdbcab80 20 client.26075 trim_cache size 0 max 16384 2024-06-02T15:50:59.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.624+0000 7f72fdbcab80 1 Processor -- start 2024-06-02T15:50:59.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.624+0000 7f72fdbcab80 1 -- start start 2024-06-02T15:50:59.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc83690 0x5650fbc84ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c97a0 con 0x5650fbc83690 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c6fd0 con 0x5650fbc7e1c0 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c4ed0 con 0x5650fbc83fb0 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72eb400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52208/0 (socket says 172.21.15.90:52208) 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72eb400700 1 -- 172.21.15.90:0/4092105964 learned_addr learned my addr 172.21.15.90:0/4092105964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.625+0000 7f72eb400700 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 -- 172.21.15.90:0/4092105964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 msgr2=0x5650fbc87700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 -- 172.21.15.90:0/4092105964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc83690 msgr2=0x5650fbc84ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc83690 0x5650fbc84ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 -- 172.21.15.90:0/4092105964 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fbc89970 con 0x5650fbc7e1c0 2024-06-02T15:50:59.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72eb400700 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7dba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72d8004790 tx=0x7f72d80011b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7304e611c127756e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72e0c00700 1 -- 172.21.15.90:0/4092105964 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8007320 con 0x5650fbc7e1c0 2024-06-02T15:50:59.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72e0c00700 1 -- 172.21.15.90:0/4092105964 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80074c0 con 0x5650fbc7e1c0 2024-06-02T15:50:59.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.626+0000 7f72e0c00700 1 -- 172.21.15.90:0/4092105964 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d801fa40 con 0x5650fbc7e1c0 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4092105964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 msgr2=0x5650fbc7dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7dba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f72d8004790 tx=0x7f72d80011b0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4092105964 shutdown_connections 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83fb0 0x5650fbc87700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc7e1c0 0x5650fbc7dba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/4092105964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc83690 0x5650fbc84ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.627+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4092105964 >> 172.21.15.90:0/4092105964 conn(0x5650fbbabc60 msgr2=0x5650fbc821e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.628+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4092105964 shutdown_connections 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.628+0000 7f72fdbcab80 1 -- 172.21.15.90:0/4092105964 wait complete. 2024-06-02T15:50:59.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.628+0000 7f72fdbcab80 1 -- start start 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.628+0000 7f72fdbcab80 1 Processor -- start 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 0x5650fbc8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c5490 con 0x5650fbc7e1c0 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c6900 con 0x5650fbc83fb0 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1c5b60 con 0x5650fbc83690 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.629+0000 7f72fdbcab80 10 client.0 _ll_register_callbacks cb 0x5650fbb75cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52216/0 (socket says 172.21.15.90:52216) 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 -- 172.21.15.90:0/1951570729 learned_addr learned my addr 172.21.15.90:0/1951570729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72eb400700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 0x5650fbc8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 -- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 msgr2=0x5650fbc8d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 0x5650fbc8d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 -- 172.21.15.90:0/1951570729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 msgr2=0x5650fbc902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:50:59.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.630+0000 7f72ebe00700 1 -- 172.21.15.90:0/1951570729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fbc8d880 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.631+0000 7f72eb400700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 0x5650fbc8d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.631+0000 7f72ebe00700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00bc80 tx=0x7f72dc005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c292151f7f0c91ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.631+0000 7f72d6000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc036060 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f72dc003860 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d80074c0 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f72dc03d050 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72fdbcab80 10 client.26017 fetch_fsmap learned FSMap version 35 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.632+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5650fbd25290 con 0x5650fbc83690 2024-06-02T15:50:59.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.633+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80074c0 con 0x5650fbc83690 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.633+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f72dc0053e0 con 0x5650fbc83690 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 10 client.26017 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 20 client.26017 populate_metadata read hostname 'smithi090' 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbd25910 0x5650fbd27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 4 client.26017 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5650fbbd75f0 con 0x5650fbd25910 2024-06-02T15:50:59.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.634+0000 7f72eb400700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbd25910 0x5650fbd27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:50:59.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.635+0000 7f72eb400700 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbd25910 0x5650fbd27d30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494d06409a7821ee server_cookie=804bfabaf746388d in_seq=0 out_seq=0 2024-06-02T15:50:59.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.635+0000 7f72d6000700 10 client.26017 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:59.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.638+0000 7f72d6000700 1 -- 172.21.15.90:0/1951570729 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5650fbbd75f0 con 0x5650fbd25910 2024-06-02T15:50:59.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.638+0000 7f72d6000700 10 client.26017 handle_command_reply: tid=0 2024-06-02T15:50:59.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.638+0000 7f72fdbcab80 1 client.26017 shutdown 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 msgr2=0x5650fbc8ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00bc80 tx=0x7f72dc005f00 comp rx=0 tx=0).stop 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 shutdown_connections 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5650fbd25910 0x5650fbd27d30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5650fbc83fb0 0x5650fbc8d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650fbc7e1c0 0x5650fbc902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 --2- 172.21.15.90:0/1951570729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650fbc83690 0x5650fbc8ccf0 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00bc80 tx=0x7f72dc005f00 comp rx=0 tx=0).stop 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 >> 172.21.15.90:0/1951570729 conn(0x5650fbbabc60 msgr2=0x5650fbc82a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72d6000700 0 client.26017 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:50:59.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.639+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 shutdown_connections 2024-06-02T15:50:59.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.640+0000 7f72fdbcab80 1 -- 172.21.15.90:0/1951570729 wait complete. 2024-06-02T15:50:59.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.640+0000 7f72fdbcab80 20 client.26017 trim_cache size 0 max 16384 2024-06-02T15:50:59.638 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:50:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:50:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "5bba5401-7f43-4eff-9382-22abf56b09a5", 2024-06-02T15:50:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:50:59.638 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.641+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 msgr2=0x7f72d00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.641+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 0x7f72d00b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72e40def00 tx=0x7f72e40e7240 comp rx=0 tx=0).stop 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.641+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 msgr2=0x7f72ec115050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.641+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f72e4007ec0 tx=0x7f72e400d3b0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 shutdown_connections 2024-06-02T15:50:59.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f72d00b2a50 0x7f72d00b4f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72ec114bd0 0x7f72ec115050 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72ec10aa80 0x7f72ec11b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 --2- 172.21.15.90:0/4226044570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72ec11cd20 0x7f72ec114690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 >> 172.21.15.90:0/4226044570 conn(0x7f72ec06e6b0 msgr2=0x7f72ec0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 shutdown_connections 2024-06-02T15:50:59.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:50:59.642+0000 7f72f3200700 1 -- 172.21.15.90:0/4226044570 wait complete. 2024-06-02T15:51:00.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:00 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:00.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:00 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:00.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:00 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:01.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:01 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:51:01.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:01 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:01.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:01 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:01.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:01 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:01.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:01 smithi090 ceph-mon[31595]: pgmap v711: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 431 op/s 2024-06-02T15:51:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:01 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:51:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:01 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:01 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:01 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:01 smithi120 ceph-mon[36451]: pgmap v711: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 431 op/s 2024-06-02T15:51:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:01 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:51:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:01 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:01 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:01 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:01 smithi135 ceph-mon[36839]: pgmap v711: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 431 op/s 2024-06-02T15:51:02.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:51:02.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:02 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:02.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:02 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:02.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:02 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:02.736 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:51:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:03 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:03 smithi120 ceph-mon[36451]: pgmap v712: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 685 op/s 2024-06-02T15:51:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:03 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:03 smithi135 ceph-mon[36839]: pgmap v712: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 685 op/s 2024-06-02T15:51:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:03 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:03 smithi090 ceph-mon[31595]: pgmap v712: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 685 op/s 2024-06-02T15:51:05.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 -- 172.21.15.90:0/1376009972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c410aa80 msgr2=0x7f42c410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 --2- 172.21.15.90:0/1376009972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c410aa80 0x7f42c410aee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009990 tx=0x7f42b0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 -- 172.21.15.90:0/1376009972 shutdown_connections 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 --2- 172.21.15.90:0/1376009972 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c41185d0 0x7f42c411a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 --2- 172.21.15.90:0/1376009972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42c410dc70 0x7f42c410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 --2- 172.21.15.90:0/1376009972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c410aa80 0x7f42c410aee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 -- 172.21.15.90:0/1376009972 >> 172.21.15.90:0/1376009972 conn(0x7f42c406ea90 msgr2=0x7f42c406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 -- 172.21.15.90:0/1376009972 shutdown_connections 2024-06-02T15:51:05.437 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:05 smithi090 ceph-mon[31595]: pgmap v713: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 740 op/s 2024-06-02T15:51:05.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.438+0000 7f42c8a00700 1 -- 172.21.15.90:0/1376009972 wait complete. 2024-06-02T15:51:05.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.440+0000 7f42c8a00700 1 Processor -- start 2024-06-02T15:51:05.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.440+0000 7f42c8a00700 1 -- start start 2024-06-02T15:51:05.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.441+0000 7f42c8a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.442+0000 7f42c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42c410dc70 0x7f42c41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.442+0000 7f42c1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.442+0000 7f42c1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58756/0 (socket says 172.21.15.90:58756) 2024-06-02T15:51:05.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.442+0000 7f42c8a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c41185d0 0x7f42c41a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c8a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f42c411e150 con 0x7f42c410dc70 2024-06-02T15:51:05.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c8a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42c411dfd0 con 0x7f42c410aa80 2024-06-02T15:51:05.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c8a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42c411e2d0 con 0x7f42c41185d0 2024-06-02T15:51:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 -- 172.21.15.90:0/3560880380 learned_addr learned my addr 172.21.15.90:0/3560880380 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 -- 172.21.15.90:0/3560880380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c41185d0 msgr2=0x7f42c41a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c41185d0 0x7f42c41a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 -- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42c410dc70 msgr2=0x7f42c41a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42c410dc70 0x7f42c41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 -- 172.21.15.90:0/3560880380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c41a9410 con 0x7f42c410aa80 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.443+0000 7f42c1e00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42b40074a0 tx=0x7f42b4007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.444+0000 7f42baa00700 1 -- 172.21.15.90:0/3560880380 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4021070 con 0x7f42c410aa80 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.444+0000 7f42baa00700 1 -- 172.21.15.90:0/3560880380 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4013ce0 con 0x7f42c410aa80 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.444+0000 7f42baa00700 1 -- 172.21.15.90:0/3560880380 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b401c700 con 0x7f42c410aa80 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.444+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c41a9700 con 0x7f42c410aa80 2024-06-02T15:51:05.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.444+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c410e0f0 con 0x7f42c410aa80 2024-06-02T15:51:05.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.451+0000 7f42baa00700 1 -- 172.21.15.90:0/3560880380 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f42b4024070 con 0x7f42c410aa80 2024-06-02T15:51:05.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.452+0000 7f42d3463b80 1 Processor -- start 2024-06-02T15:51:05.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.452+0000 7f42d3463b80 1 -- start start 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.453+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.453+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604eeeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.453+0000 7f42baa00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 0x7f42a80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.453+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604eef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.453+0000 7f42c2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33618/0 (socket says 172.21.15.90:33618) 2024-06-02T15:51:05.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42d3463b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557604e4ce40 con 0x557604eef140 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42d3463b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557604e3c9e0 con 0x557604ef4f30 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42d3463b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557604e40a20 con 0x557604ef4610 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 -- 172.21.15.90:0/2548330236 learned_addr learned my addr 172.21.15.90:0/2548330236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604eef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 -- 172.21.15.90:0/2548330236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 msgr2=0x557604eeeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604eeeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 -- 172.21.15.90:0/2548330236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 msgr2=0x557604eef5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604eef5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.454+0000 7f42c2800700 1 -- 172.21.15.90:0/2548330236 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557604e05420 con 0x557604ef4610 2024-06-02T15:51:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.455+0000 7f42c2800700 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9897bb3640ab6952 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.455+0000 7f42baa00700 1 -- 172.21.15.90:0/3560880380 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f42b40ad5d0 con 0x7f42c410aa80 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.455+0000 7f42af400700 1 -- 172.21.15.90:0/2548330236 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc014040 con 0x557604ef4610 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.455+0000 7f42af400700 1 -- 172.21.15.90:0/2548330236 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42bc004db0 con 0x557604ef4610 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.455+0000 7f42af400700 1 -- 172.21.15.90:0/2548330236 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc01b6e0 con 0x557604ef4610 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.456+0000 7f42c1400700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 0x7f42a80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.456+0000 7f42d3463b80 1 -- 172.21.15.90:0/2548330236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 msgr2=0x557604ef49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.456+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc009e60 comp rx=0 tx=0).stop 2024-06-02T15:51:05.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.457+0000 7f42d3463b80 1 -- 172.21.15.90:0/2548330236 shutdown_connections 2024-06-02T15:51:05.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.457+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604eef5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.457+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4610 0x557604ef49f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.457+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2548330236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604eeeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.457+0000 7f42d3463b80 1 -- 172.21.15.90:0/2548330236 >> 172.21.15.90:0/2548330236 conn(0x557604e1cbe0 msgr2=0x557604ee9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.458+0000 7f42d3463b80 1 -- 172.21.15.90:0/2548330236 shutdown_connections 2024-06-02T15:51:05.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.459+0000 7f42d3463b80 1 -- 172.21.15.90:0/2548330236 wait complete. 2024-06-02T15:51:05.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.459+0000 7f42d3463b80 1 -- start start 2024-06-02T15:51:05.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.459+0000 7f42d3463b80 1 Processor -- start 2024-06-02T15:51:05.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.460+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604efadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.460+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 0x557604ef81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.460+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42d3463b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557604d42fa0 con 0x557604eef140 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42d3463b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557604ddd760 con 0x557604ef4610 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 0x557604ef81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42d3463b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557604f1d9f0 con 0x557604ef4f30 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42d3463b80 10 client.0 _ll_register_callbacks cb 0x557604de6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 0x557604ef81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58768/0 (socket says 172.21.15.90:58768) 2024-06-02T15:51:05.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42c1400700 1 -- 172.21.15.90:0/1595339430 learned_addr learned my addr 172.21.15.90:0/1595339430 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:05.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42c1e00700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604efadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.461+0000 7f42c2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1400700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 0x7f42a80b4e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009350 tx=0x7f42b000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1e00700 1 -- 172.21.15.90:0/1595339430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 msgr2=0x557604ef86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1e00700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1e00700 1 -- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 msgr2=0x557604ef81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1e00700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 0x557604ef81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.462+0000 7f42c1e00700 1 -- 172.21.15.90:0/1595339430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557604d8c8d0 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.463+0000 7f42c1e00700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604efadc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42b40ab660 tx=0x7f42b40e7390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3859209de71c25 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.463+0000 7f42a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.463+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40ad7d0 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.463+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42bc004db0 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.463+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f42b40b9bb0 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f42b40ca070 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42d3463b80 10 client.15876 fetch_fsmap learned FSMap version 36 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557604e0d060 con 0x557604eef140 2024-06-02T15:51:05.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42bc004db0 con 0x557604eef140 2024-06-02T15:51:05.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f42b40b0750 con 0x557604eef140 2024-06-02T15:51:05.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42d3463b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 36 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42d3463b80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.464+0000 7f42d3463b80 20 client.15876 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.465+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f2a480 0x557604f2c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.465+0000 7f42d3463b80 4 client.15876 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.465+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557604e4ed90 con 0x557604f2a480 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.465+0000 7f42c1400700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f2a480 0x557604f2c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.466+0000 7f42c1400700 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f2a480 0x557604f2c870 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81536ce012d668f3 server_cookie=2ba248fc2842bce6 in_seq=0 out_seq=0 2024-06-02T15:51:05.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.466+0000 7f42a3e00700 10 client.15876 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:05.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.468+0000 7f42a3e00700 1 -- 172.21.15.90:0/1595339430 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557604e4ed90 con 0x557604f2a480 2024-06-02T15:51:05.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.468+0000 7f42a3e00700 10 client.15876 handle_command_reply: tid=0 2024-06-02T15:51:05.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.468+0000 7f42d3463b80 1 client.15876 shutdown 2024-06-02T15:51:05.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.468+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 msgr2=0x557604efadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.468+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604efadc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f42b40ab660 tx=0x7f42b40e7390 comp rx=0 tx=0).stop 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 shutdown_connections 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f2a480 0x557604f2c870 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42a3e00700 0 client.15876 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604eef140 0x557604efadc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 --2- 172.21.15.90:0/1595339430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4610 0x557604ef81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 >> 172.21.15.90:0/1595339430 conn(0x557604e1cbe0 msgr2=0x557604ee8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 shutdown_connections 2024-06-02T15:51:05.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 1 -- 172.21.15.90:0/1595339430 wait complete. 2024-06-02T15:51:05.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.469+0000 7f42d3463b80 20 client.15876 trim_cache size 0 max 16384 2024-06-02T15:51:05.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.516+0000 7f42d3463b80 1 Processor -- start 2024-06-02T15:51:05.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.516+0000 7f42d3463b80 1 -- start start 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.516+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604ee94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42d3463b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f42c407c930 con 0x557604ef4610 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42d3463b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42c407a130 con 0x557604eef140 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42d3463b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4077fc0 con 0x557604ef4f30 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58772/0 (socket says 172.21.15.90:58772) 2024-06-02T15:51:05.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 -- 172.21.15.90:0/938699783 learned_addr learned my addr 172.21.15.90:0/938699783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:05.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c2800700 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1400700 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604ee94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 -- 172.21.15.90:0/938699783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 msgr2=0x557604ef9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 -- 172.21.15.90:0/938699783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 msgr2=0x557604ee94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.517+0000 7f42c1e00700 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604ee94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.518+0000 7f42c1e00700 1 -- 172.21.15.90:0/938699783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557604f2cef0 con 0x557604eef140 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.518+0000 7f42c1e00700 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42b4007c60 tx=0x7f42b4014980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a719072a1bfab8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.518+0000 7f42aea00700 1 -- 172.21.15.90:0/938699783 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40ad970 con 0x557604eef140 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.518+0000 7f42aea00700 1 -- 172.21.15.90:0/938699783 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40b99b0 con 0x557604eef140 2024-06-02T15:51:05.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.518+0000 7f42aea00700 1 -- 172.21.15.90:0/938699783 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40adb40 con 0x557604eef140 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 -- 172.21.15.90:0/938699783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 msgr2=0x557604f26a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f42b4007c60 tx=0x7f42b4014980 comp rx=0 tx=0).stop 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 -- 172.21.15.90:0/938699783 shutdown_connections 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604ef4f30 0x557604ef9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604eef140 0x557604f26a90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 --2- 172.21.15.90:0/938699783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604ee94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.519+0000 7f42d3463b80 1 -- 172.21.15.90:0/938699783 >> 172.21.15.90:0/938699783 conn(0x557604e1cbe0 msgr2=0x557604ef38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.520+0000 7f42d3463b80 1 -- 172.21.15.90:0/938699783 shutdown_connections 2024-06-02T15:51:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.520+0000 7f42d3463b80 1 -- 172.21.15.90:0/938699783 wait complete. 2024-06-02T15:51:05.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 -- start start 2024-06-02T15:51:05.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 Processor -- start 2024-06-02T15:51:05.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604eef140 0x557604f23140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604f1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4078610 con 0x557604ef4610 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4079a60 con 0x557604ef4f30 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.521+0000 7f42d3463b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4078cc0 con 0x557604eef140 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42d3463b80 10 client.0 _ll_register_callbacks cb 0x557604de6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58776/0 (socket says 172.21.15.90:58776) 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 -- 172.21.15.90:0/2043780179 learned_addr learned my addr 172.21.15.90:0/2043780179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 -- 172.21.15.90:0/2043780179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604eef140 msgr2=0x557604f23140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604eef140 0x557604f23140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.522+0000 7f42c2800700 1 -- 172.21.15.90:0/2043780179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 msgr2=0x557604f1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.523+0000 7f42c1400700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604f1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.523+0000 7f42c2800700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604f1fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.523+0000 7f42c2800700 1 -- 172.21.15.90:0/2043780179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557604f207d0 con 0x557604ef4f30 2024-06-02T15:51:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.523+0000 7f42c1400700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604f1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42c2800700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e503131845ac89fb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc02b060 con 0x557604ef4f30 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b40b99b0 con 0x557604ef4f30 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f42bc0156e0 con 0x557604ef4f30 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f42bc032050 con 0x557604ef4f30 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42d3463b80 10 client.26096 fetch_fsmap learned FSMap version 36 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.524+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557604f95e80 con 0x557604ef4f30 2024-06-02T15:51:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40b99b0 con 0x557604ef4f30 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f42bc0045c0 con 0x557604ef4f30 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 10 client.26096 fetch_fsmap finished waiting for FSMap version 36 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 20 client.26096 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f96370 0x557604f98790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 4 client.26096 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:51:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.525+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557604e48570 con 0x557604f96370 2024-06-02T15:51:05.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.526+0000 7f42c1e00700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f96370 0x557604f98790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:05.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.527+0000 7f42c1e00700 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f96370 0x557604f98790 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d937f71a6ff492 server_cookie=f36d2e1bbc2b73ed in_seq=0 out_seq=0 2024-06-02T15:51:05.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.527+0000 7f42a3e00700 10 client.26096 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:05.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.527+0000 7f42a3e00700 1 -- 172.21.15.90:0/2043780179 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557604e48570 con 0x557604f96370 2024-06-02T15:51:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.527+0000 7f42a3e00700 10 client.26096 handle_command_reply: tid=0 2024-06-02T15:51:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 client.26096 shutdown 2024-06-02T15:51:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 msgr2=0x557604f20190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009fd0 tx=0x7f42bc009d90 comp rx=0 tx=0).stop 2024-06-02T15:51:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 shutdown_connections 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557604f96370 0x557604f98790 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557604ef4f30 0x557604f20190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557604eef140 0x557604f23140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42a3e00700 0 client.26096 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.528+0000 7f42d3463b80 1 --2- 172.21.15.90:0/2043780179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557604ef4610 0x557604f1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.529+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 >> 172.21.15.90:0/2043780179 conn(0x557604e1cbe0 msgr2=0x557604ef9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.529+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 shutdown_connections 2024-06-02T15:51:05.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.529+0000 7f42d3463b80 1 -- 172.21.15.90:0/2043780179 wait complete. 2024-06-02T15:51:05.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.530+0000 7f42d3463b80 20 client.26096 trim_cache size 0 max 16384 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "5bba5401-7f43-4eff-9382-22abf56b09a5": { 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "5bba5401-7f43-4eff-9382-22abf56b09a5", 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:51:05.528 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:51:05.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 msgr2=0x7f42a80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 0x7f42a80b4e90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009350 tx=0x7f42b000b260 comp rx=0 tx=0).stop 2024-06-02T15:51:05.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 msgr2=0x7f42c41a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:05.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42b40074a0 tx=0x7f42b4007860 comp rx=0 tx=0).stop 2024-06-02T15:51:05.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 shutdown_connections 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f42a80b29d0 0x7f42a80b4e90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42c41185d0 0x7f42c41a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42c410aa80 0x7f42c41a4400 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 --2- 172.21.15.90:0/3560880380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42c410dc70 0x7f42c41a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.532+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 >> 172.21.15.90:0/3560880380 conn(0x7f42c406ea90 msgr2=0x7f42c4119bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.533+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 shutdown_connections 2024-06-02T15:51:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:05.533+0000 7f42c8a00700 1 -- 172.21.15.90:0/3560880380 wait complete. 2024-06-02T15:51:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:05 smithi120 ceph-mon[36451]: pgmap v713: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 740 op/s 2024-06-02T15:51:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:05 smithi135 ceph-mon[36839]: pgmap v713: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.2 MiB/s wr, 740 op/s 2024-06-02T15:51:06.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:06 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:06.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:06 smithi120 ceph-mon[36451]: pgmap v714: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.33k op/s 2024-06-02T15:51:06.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:06 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:06.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:06 smithi135 ceph-mon[36839]: pgmap v714: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.33k op/s 2024-06-02T15:51:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:06 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:06 smithi090 ceph-mon[31595]: pgmap v714: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.33k op/s 2024-06-02T15:51:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 -- 172.21.15.90:0/3081483669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c077410 msgr2=0x7f1f8c077830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 --2- 172.21.15.90:0/3081483669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c077410 0x7f1f8c077830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009900 tx=0x7f1f7802a3f0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 -- 172.21.15.90:0/3081483669 shutdown_connections 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 --2- 172.21.15.90:0/3081483669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c079890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 --2- 172.21.15.90:0/3081483669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078650 0x7f1f8c078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 --2- 172.21.15.90:0/3081483669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c077410 0x7f1f8c077830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 -- 172.21.15.90:0/3081483669 >> 172.21.15.90:0/3081483669 conn(0x7f1f8c100080 msgr2=0x7f1f8c1024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 -- 172.21.15.90:0/3081483669 shutdown_connections 2024-06-02T15:51:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.357+0000 7f1f8b200640 1 -- 172.21.15.90:0/3081483669 wait complete. 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 Processor -- start 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 -- start start 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 0x7f1f8c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c1a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114a90 con 0x7f1f8c078ff0 2024-06-02T15:51:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114910 con 0x7f1f8c077410 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.359+0000 7f1f8b200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114c10 con 0x7f1f8c078650 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58788/0 (socket says 172.21.15.90:58788) 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89e00640 1 -- 172.21.15.90:0/1858740649 learned_addr learned my addr 172.21.15.90:0/1858740649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 0x7f1f8c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f8a800640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c1a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 -- 172.21.15.90:0/1858740649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 msgr2=0x7f1f8c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 -- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 msgr2=0x7f1f8c1a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c1a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89400640 1 -- 172.21.15.90:0/1858740649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1a6cf0 con 0x7f1f8c078650 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f89e00640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:51:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.360+0000 7f1f8a800640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c1a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f89400640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 0x7f1f8c19f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00d6d0 tx=0x7f1f7c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f82a00640 1 -- 172.21.15.90:0/1858740649 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c017070 con 0x7f1f8c078650 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1a6fe0 con 0x7f1f8c078650 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f82a00640 1 -- 172.21.15.90:0/1858740649 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c009820 con 0x7f1f8c078650 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f82a00640 1 -- 172.21.15.90:0/1858740649 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c004c40 con 0x7f1f8c078650 2024-06-02T15:51:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.361+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a74a0 con 0x7f1f8c078650 2024-06-02T15:51:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 Processor -- start 2024-06-02T15:51:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 -- start start 2024-06-02T15:51:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c48990c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c48990ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c489876b90 con 0x55c48990c870 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.365+0000 7f1f93def740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c489866730 con 0x55c489906d10 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f93def740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c48986a770 con 0x55c489907700 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58794/0 (socket says 172.21.15.90:58794) 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 -- 172.21.15.90:0/255388432 learned_addr learned my addr 172.21.15.90:0/255388432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f89e00640 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c48990c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f82a00640 1 -- 172.21.15.90:0/1858740649 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f7c005220 con 0x7f1f8c078650 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 -- 172.21.15.90:0/255388432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 msgr2=0x55c48990c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c48990c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 -- 172.21.15.90:0/255388432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 msgr2=0x55c48990ed00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:51:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c48990ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.366+0000 7f1f8a800640 1 -- 172.21.15.90:0/255388432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4897ff8b0 con 0x55c489906d10 2024-06-02T15:51:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.367+0000 7f1f8a800640 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009fd0 tx=0x7f1f840108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec4d3c03d4335f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.367+0000 7f1f77400640 1 -- 172.21.15.90:0/255388432 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011760 con 0x55c489906d10 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.367+0000 7f1f77400640 1 -- 172.21.15.90:0/255388432 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84011db0 con 0x55c489906d10 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.367+0000 7f1f77400640 1 -- 172.21.15.90:0/255388432 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8400e3c0 con 0x55c489906d10 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.368+0000 7f1f93def740 1 -- 172.21.15.90:0/255388432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 msgr2=0x55c489907130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.368+0000 7f1f93def740 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009fd0 tx=0x7f1f840108e0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.368+0000 7f1f82a00640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 0x7f1f700b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.368+0000 7f1f82a00640 1 -- 172.21.15.90:0/1858740649 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f7c0e2100 con 0x7f1f8c078650 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 -- 172.21.15.90:0/255388432 shutdown_connections 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c48990ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c48990c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 --2- 172.21.15.90:0/255388432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489907130 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 -- 172.21.15.90:0/255388432 >> 172.21.15.90:0/255388432 conn(0x55c489902380 msgr2=0x55c489902780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f89e00640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 0x7f1f700b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 -- 172.21.15.90:0/255388432 shutdown_connections 2024-06-02T15:51:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.369+0000 7f1f93def740 1 -- 172.21.15.90:0/255388432 wait complete. 2024-06-02T15:51:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.370+0000 7f1f93def740 1 -- start start 2024-06-02T15:51:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.370+0000 7f1f89e00640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 0x7f1f700b4ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c489908170 tx=0x7f1f78005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.370+0000 7f1f93def740 1 Processor -- start 2024-06-02T15:51:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c489906d10 0x55c489910b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48990c870 0x55c489918080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c48954c790 con 0x55c489906d10 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c48980ebc0 con 0x55c489907700 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c48990f820 con 0x55c48990c870 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f93def740 10 client.0 _ll_register_callbacks cb 0x55c4897ec400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58804/0 (socket says 172.21.15.90:58804) 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f89400640 1 -- 172.21.15.90:0/2267728269 learned_addr learned my addr 172.21.15.90:0/2267728269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.371+0000 7f1f89e00640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c489906d10 0x55c489910b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 -- 172.21.15.90:0/2267728269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48990c870 msgr2=0x55c489918080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48990c870 0x55c489918080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 -- 172.21.15.90:0/2267728269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c489906d10 msgr2=0x55c489910b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c489906d10 0x55c489910b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 -- 172.21.15.90:0/2267728269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c48957b210 con 0x55c489907700 2024-06-02T15:51:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f89400640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0a6580 tx=0x7f1f7c0a8f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39395991e36a5833 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f6be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c020070 con 0x55c489907700 2024-06-02T15:51:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f7c0099d0 con 0x55c489907700 2024-06-02T15:51:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.372+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f84011db0 con 0x55c489907700 2024-06-02T15:51:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.373+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c0ad6c0 con 0x55c489907700 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.373+0000 7f1f93def740 10 client.26108 fetch_fsmap learned FSMap version 36 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c489647620 con 0x55c489907700 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84011db0 con 0x55c489907700 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f1f7c0ad150 con 0x55c489907700 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 10 client.26108 fetch_fsmap finished waiting for FSMap version 36 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 20 client.26108 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c48991c680 0x55c48991ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 4 client.26108 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.374+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c489878ae0 con 0x55c48991c680 2024-06-02T15:51:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.375+0000 7f1f89e00640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c48991c680 0x55c48991ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.375+0000 7f1f89e00640 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c48991c680 0x55c48991ea90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9aca4aae4452bb server_cookie=f46c4dc8131fa0aa in_seq=0 out_seq=0 2024-06-02T15:51:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.375+0000 7f1f6be00640 10 client.26108 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.376+0000 7f1f6be00640 1 -- 172.21.15.90:0/2267728269 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c489878ae0 con 0x55c48991c680 2024-06-02T15:51:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f6be00640 10 client.26108 handle_command_reply: tid=0 2024-06-02T15:51:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 client.26108 shutdown 2024-06-02T15:51:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 msgr2=0x55c489911040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0a6580 tx=0x7f1f7c0a8f60 comp rx=0 tx=0).stop 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 shutdown_connections 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c48991c680 0x55c48991ea90 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c48990c870 0x55c489918080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489911040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 --2- 172.21.15.90:0/2267728269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c489906d10 0x55c489910b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f6be00640 0 client.26108 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.377+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 >> 172.21.15.90:0/2267728269 conn(0x55c489902380 msgr2=0x55c4899013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.378+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 shutdown_connections 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.378+0000 7f1f93def740 1 -- 172.21.15.90:0/2267728269 wait complete. 2024-06-02T15:51:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.378+0000 7f1f93def740 20 client.26108 trim_cache size 0 max 16384 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.395+0000 7f1f93def740 1 Processor -- start 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 -- start start 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489901060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b43d0 con 0x55c48990c870 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b1c10 con 0x55c489906d10 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f93def740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1afa60 con 0x55c489907700 2024-06-02T15:51:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f89400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f89e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58814/0 (socket says 172.21.15.90:58814) 2024-06-02T15:51:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f89e00640 1 -- 172.21.15.90:0/1701830895 learned_addr learned my addr 172.21.15.90:0/1701830895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f89400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:33692/0 (socket says 172.21.15.90:33692) 2024-06-02T15:51:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.396+0000 7f1f8a800640 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489901060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 -- 172.21.15.90:0/1701830895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 msgr2=0x55c489900b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 -- 172.21.15.90:0/1701830895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 msgr2=0x55c489901060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489901060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 -- 172.21.15.90:0/1701830895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c48990fbc0 con 0x55c489907700 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89e00640 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f89400640 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00d6a0 tx=0x7f1f7c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3464f81950257cea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f76a00640 1 -- 172.21.15.90:0/1701830895 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c020070 con 0x55c489907700 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f76a00640 1 -- 172.21.15.90:0/1701830895 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c0b2170 con 0x55c489907700 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.397+0000 7f1f76a00640 1 -- 172.21.15.90:0/1701830895 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0a55a0 con 0x55c489907700 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 -- 172.21.15.90:0/1701830895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 msgr2=0x55c489918040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00d6a0 tx=0x7f1f7c000bc0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 -- 172.21.15.90:0/1701830895 shutdown_connections 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489901060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489907700 0x55c489918040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 --2- 172.21.15.90:0/1701830895 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489906d10 0x55c489900b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 -- 172.21.15.90:0/1701830895 >> 172.21.15.90:0/1701830895 conn(0x55c489902380 msgr2=0x55c489908a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.398+0000 7f1f93def740 1 -- 172.21.15.90:0/1701830895 shutdown_connections 2024-06-02T15:51:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.399+0000 7f1f93def740 1 -- 172.21.15.90:0/1701830895 wait complete. 2024-06-02T15:51:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.399+0000 7f1f93def740 1 -- start start 2024-06-02T15:51:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.399+0000 7f1f93def740 1 Processor -- start 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.399+0000 7f1f93def740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489906d10 0x55c489915010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.399+0000 7f1f93def740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f93def740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489985260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f93def740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b00d0 con 0x55c48990c870 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f93def740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b1540 con 0x55c489907700 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f93def740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b07a0 con 0x55c489906d10 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f93def740 10 client.0 _ll_register_callbacks cb 0x7f1f7c0057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58820/0 (socket says 172.21.15.90:58820) 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 -- 172.21.15.90:0/867276163 learned_addr learned my addr 172.21.15.90:0/867276163 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f8a800640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489985260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89e00640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489906d10 0x55c489915010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 -- 172.21.15.90:0/867276163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489906d10 msgr2=0x55c489915010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489906d10 0x55c489915010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 -- 172.21.15.90:0/867276163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 msgr2=0x55c489985260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489985260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 -- 172.21.15.90:0/867276163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c489985850 con 0x55c489907700 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f89400640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0c5050 tx=0x7f1f7c0a8ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec404a3c3095edd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.400+0000 7f1f6be00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c020070 con 0x55c489907700 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f7c0a5900 con 0x55c489907700 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7c0b2170 con 0x55c489907700 2024-06-02T15:51:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c0e4390 con 0x55c489907700 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f93def740 10 client.26114 fetch_fsmap learned FSMap version 36 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c489985850 con 0x55c489907700 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c0b2170 con 0x55c489907700 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f1f7c0ae850 con 0x55c489907700 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f93def740 10 client.26114 fetch_fsmap finished waiting for FSMap version 36 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.401+0000 7f1f93def740 10 client.26114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f93def740 20 client.26114 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c489987140 0x55c489989560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f93def740 4 client.26114 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4898722c0 con 0x55c489987140 2024-06-02T15:51:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f89e00640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c489987140 0x55c489989560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f89e00640 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c489987140 0x55c489989560 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac1a81f55041b4c server_cookie=9aaa7c6f72f770ec in_seq=0 out_seq=0 2024-06-02T15:51:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:08.402+0000 7f1f6be00640 10 client.26114 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:09 smithi120 ceph-mon[36451]: pgmap v715: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 1.20k op/s 2024-06-02T15:51:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:09 smithi135 ceph-mon[36839]: pgmap v715: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 1.20k op/s 2024-06-02T15:51:09.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:09 smithi090 ceph-mon[31595]: pgmap v715: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 1.20k op/s 2024-06-02T15:51:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.843+0000 7f1f6be00640 1 -- 172.21.15.90:0/867276163 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c4898722c0 con 0x55c489987140 2024-06-02T15:51:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.843+0000 7f1f6be00640 10 client.26114 handle_command_reply: tid=0 2024-06-02T15:51:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.845+0000 7f1f93def740 1 client.26114 shutdown 2024-06-02T15:51:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.845+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 msgr2=0x55c489915550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.845+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0c5050 tx=0x7f1f7c0a8ba0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.845+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 shutdown_connections 2024-06-02T15:51:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.845+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55c489987140 0x55c489989560 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f6be00640 0 client.26114 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c48990c870 0x55c489985260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c489907700 0x55c489915550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 --2- 172.21.15.90:0/867276163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c489906d10 0x55c489915010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 >> 172.21.15.90:0/867276163 conn(0x55c489902380 msgr2=0x55c489905060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 shutdown_connections 2024-06-02T15:51:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.846+0000 7f1f93def740 1 -- 172.21.15.90:0/867276163 wait complete. 2024-06-02T15:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.847+0000 7f1f93def740 20 client.26114 trim_cache size 0 max 16384 2024-06-02T15:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:51:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.848+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 msgr2=0x7f1f700b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.848+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 0x7f1f700b4ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c489908170 tx=0x7f1f78005a20 comp rx=0 tx=0).stop 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.848+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 msgr2=0x7f1f8c19f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.848+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 0x7f1f8c19f610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00d6d0 tx=0x7f1f7c00db90 comp rx=0 tx=0).stop 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.848+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 shutdown_connections 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1f700b27e0 0x7f1f700b4ca0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f8c078ff0 0x7f1f8c1a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c078650 0x7f1f8c19f610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 --2- 172.21.15.90:0/1858740649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c077410 0x7f1f8c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 >> 172.21.15.90:0/1858740649 conn(0x7f1f8c100080 msgr2=0x7f1f8c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 shutdown_connections 2024-06-02T15:51:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:09.849+0000 7f1f8b200640 1 -- 172.21.15.90:0/1858740649 wait complete. 2024-06-02T15:51:10.053 INFO:tasks.cephfs.filesystem:scrub status for tag:5bba5401-7f43-4eff-9382-22abf56b09a5 - {'path': '/', 'tag': '5bba5401-7f43-4eff-9382-22abf56b09a5', 'options': 'recursive,force'} 2024-06-02T15:51:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:11 smithi120 ceph-mon[36451]: pgmap v716: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.20k op/s 2024-06-02T15:51:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:11 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:11 smithi135 ceph-mon[36839]: pgmap v716: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.20k op/s 2024-06-02T15:51:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:11 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:11 smithi090 ceph-mon[31595]: pgmap v716: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 1.20k op/s 2024-06-02T15:51:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:11 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:51:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:13 smithi120 ceph-mon[36451]: pgmap v717: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 1.36k op/s 2024-06-02T15:51:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:13 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:51:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:13 smithi135 ceph-mon[36839]: pgmap v717: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 1.36k op/s 2024-06-02T15:51:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:13 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:51:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:13 smithi090 ceph-mon[31595]: pgmap v717: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 1.36k op/s 2024-06-02T15:51:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:13 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:51:15.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:15.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:15.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: pgmap v718: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: pgmap v718: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:15 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: pgmap v718: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 1.16k op/s 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:15 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:15.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:15 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:16 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:16.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:16 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:16 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:17 smithi120 ceph-mon[36451]: pgmap v719: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.36k op/s 2024-06-02T15:51:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:17 smithi135 ceph-mon[36839]: pgmap v719: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.36k op/s 2024-06-02T15:51:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:17 smithi090 ceph-mon[31595]: pgmap v719: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 1.36k op/s 2024-06-02T15:51:18.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:18 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:18.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:18 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:18.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:18 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:19.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:19 smithi120 ceph-mon[36451]: pgmap v720: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 853 op/s 2024-06-02T15:51:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:19 smithi135 ceph-mon[36839]: pgmap v720: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 853 op/s 2024-06-02T15:51:19.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:19 smithi090 ceph-mon[31595]: pgmap v720: 129 pgs: 129 active+clean; 156 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 853 op/s 2024-06-02T15:51:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:21 smithi120 ceph-mon[36451]: pgmap v721: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 866 op/s 2024-06-02T15:51:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:21 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:21 smithi135 ceph-mon[36839]: pgmap v721: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 866 op/s 2024-06-02T15:51:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:21 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:21 smithi090 ceph-mon[31595]: pgmap v721: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 866 op/s 2024-06-02T15:51:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:21 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 -- 172.21.15.90:0/852580999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0075450 msgr2=0x7f43f00758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 --2- 172.21.15.90:0/852580999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0075450 0x7f43f00758b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009900 tx=0x7f43dc02a480 comp rx=0 tx=0).stop 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 -- 172.21.15.90:0/852580999 shutdown_connections 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 --2- 172.21.15.90:0/852580999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f010a940 0x7f43f010cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 --2- 172.21.15.90:0/852580999 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0075450 0x7f43f00758b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 --2- 172.21.15.90:0/852580999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f0076a30 0x7f43f0074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.098+0000 7f43f7c00640 1 -- 172.21.15.90:0/852580999 >> 172.21.15.90:0/852580999 conn(0x7f43f00fe040 msgr2=0x7f43f0100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.099+0000 7f43f7c00640 1 -- 172.21.15.90:0/852580999 shutdown_connections 2024-06-02T15:51:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.099+0000 7f43f7c00640 1 -- 172.21.15.90:0/852580999 wait complete. 2024-06-02T15:51:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.100+0000 7f43f7c00640 1 Processor -- start 2024-06-02T15:51:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 -- start start 2024-06-02T15:51:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f0075450 0x7f43f019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 0x7f43f01a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43f0110850 con 0x7f43f010a940 2024-06-02T15:51:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01106d0 con 0x7f43f0076a30 2024-06-02T15:51:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.102+0000 7f43f7c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01109d0 con 0x7f43f0075450 2024-06-02T15:51:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.103+0000 7f43efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.103+0000 7f43efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50778/0 (socket says 172.21.15.90:50778) 2024-06-02T15:51:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.103+0000 7f43efe00640 1 -- 172.21.15.90:0/3487275322 learned_addr learned my addr 172.21.15.90:0/3487275322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.103+0000 7f43f5200640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f0075450 0x7f43f019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43f5c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 0x7f43f01a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43efe00640 1 -- 172.21.15.90:0/3487275322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f0075450 msgr2=0x7f43f019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43efe00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f0075450 0x7f43f019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43efe00640 1 -- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 msgr2=0x7f43f01a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43efe00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 0x7f43f01a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43efe00640 1 -- 172.21.15.90:0/3487275322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f0102540 con 0x7f43f0076a30 2024-06-02T15:51:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.104+0000 7f43f5c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 0x7f43f01a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:51:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.105+0000 7f43efe00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005940 tx=0x7f43dc004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.106+0000 7f43ed600640 1 -- 172.21.15.90:0/3487275322 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc03f030 con 0x7f43f0076a30 2024-06-02T15:51:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.106+0000 7f43ed600640 1 -- 172.21.15.90:0/3487275322 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc003cb0 con 0x7f43f0076a30 2024-06-02T15:51:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.106+0000 7f43ed600640 1 -- 172.21.15.90:0/3487275322 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc035070 con 0x7f43f0076a30 2024-06-02T15:51:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.106+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01027d0 con 0x7f43f0076a30 2024-06-02T15:51:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.107+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f0102cf0 con 0x7f43f0076a30 2024-06-02T15:51:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.110+0000 7f43fa9b2740 1 Processor -- start 2024-06-02T15:51:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 -- start start 2024-06-02T15:51:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfcff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfd7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5620ecf3fb90 con 0x5620ecfd5690 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620ecf2f730 con 0x5620ecfcfb30 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.111+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5620ecf33770 con 0x5620ecfd0520 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfcff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44362/0 (socket says 172.21.15.90:44362) 2024-06-02T15:51:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5200640 1 -- 172.21.15.90:0/1727650669 learned_addr learned my addr 172.21.15.90:0/1727650669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43ed600640 1 -- 172.21.15.90:0/3487275322 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f43dc042030 con 0x7f43f0076a30 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5c00640 1 -- 172.21.15.90:0/1727650669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 msgr2=0x5620ecfd5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5c00640 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.112+0000 7f43f5c00640 1 -- 172.21.15.90:0/1727650669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 msgr2=0x5620ecfd7b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.113+0000 7f43f5c00640 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfd7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.113+0000 7f43f5c00640 1 -- 172.21.15.90:0/1727650669 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ecec88b0 con 0x5620ecfcfb30 2024-06-02T15:51:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.114+0000 7f43f5c00640 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfcff50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009ee0 tx=0x7f43e8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa1c4e8ec878f0b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.114+0000 7f43e2000640 1 -- 172.21.15.90:0/1727650669 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8016070 con 0x5620ecfcfb30 2024-06-02T15:51:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.114+0000 7f43e2000640 1 -- 172.21.15.90:0/1727650669 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e8012900 con 0x5620ecfcfb30 2024-06-02T15:51:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43ed600640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 0x7f43d80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1727650669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 msgr2=0x5620ecfcff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfcff50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009ee0 tx=0x7f43e8010740 comp rx=0 tx=0).stop 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43ed600640 1 -- 172.21.15.90:0/3487275322 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f43dc102b00 con 0x7f43f0076a30 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1727650669 shutdown_connections 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfd7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1727650669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfcff50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.115+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1727650669 >> 172.21.15.90:0/1727650669 conn(0x5620ecfcb1a0 msgr2=0x5620ecfcb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.117+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1727650669 shutdown_connections 2024-06-02T15:51:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.118+0000 7f43f5200640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 0x7f43d80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.119+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1727650669 wait complete. 2024-06-02T15:51:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.120+0000 7f43f5200640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 0x7f43d80b4eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5620ecfcc070 tx=0x7f43e4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.120+0000 7f43fa9b2740 1 -- start start 2024-06-02T15:51:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.120+0000 7f43fa9b2740 1 Processor -- start 2024-06-02T15:51:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfe0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5620ecc15790 con 0x5620ecfd5690 2024-06-02T15:51:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5620eced7bc0 con 0x5620ecfcfb30 2024-06-02T15:51:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5620ecfd86e0 con 0x5620ecfd0520 2024-06-02T15:51:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.121+0000 7f43fa9b2740 10 client.0 _ll_register_callbacks cb 0x5620eceb5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50790/0 (socket says 172.21.15.90:50790) 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 -- 172.21.15.90:0/3938821066 learned_addr learned my addr 172.21.15.90:0/3938821066 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 -- 172.21.15.90:0/3938821066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 msgr2=0x5620ecfd9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43efe00640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 -- 172.21.15.90:0/3938821066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 msgr2=0x5620ecfe0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfe0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43f5200640 1 -- 172.21.15.90:0/3938821066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ecc44210 con 0x5620ecfcfb30 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.122+0000 7f43efe00640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.123+0000 7f43f5200640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43e40149c0 tx=0x7f43e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588c3089555dc5ac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.123+0000 7f43d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.123+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4025070 con 0x5620ecfcfb30 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.123+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f43e4021600 con 0x5620ecfcfb30 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.123+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43e8012900 con 0x5620ecfcfb30 2024-06-02T15:51:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.124+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f43e402f050 con 0x5620ecfcfb30 2024-06-02T15:51:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.126+0000 7f43fa9b2740 10 client.26126 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.126+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5620ecd10620 con 0x5620ecfcfb30 2024-06-02T15:51:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e8012900 con 0x5620ecfcfb30 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f43e4020070 con 0x5620ecfcfb30 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 10 client.26126 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 20 client.26126 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ecfe55f0 0x5620ecfe7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 4 client.26126 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.127+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620ecf41ae0 con 0x5620ecfe55f0 2024-06-02T15:51:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.128+0000 7f43efe00640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ecfe55f0 0x5620ecfe7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.128+0000 7f43efe00640 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ecfe55f0 0x5620ecfe7a40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f3be630ae35e56 server_cookie=31338f6a1b4b6868 in_seq=0 out_seq=0 2024-06-02T15:51:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.128+0000 7f43d6a00640 10 client.26126 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.129+0000 7f43d6a00640 1 -- 172.21.15.90:0/3938821066 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5620ecf41ae0 con 0x5620ecfe55f0 2024-06-02T15:51:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.129+0000 7f43d6a00640 10 client.26126 handle_command_reply: tid=0 2024-06-02T15:51:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.129+0000 7f43fa9b2740 1 client.26126 shutdown 2024-06-02T15:51:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.129+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 msgr2=0x5620ecfd99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.129+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f43e40149c0 tx=0x7f43e40046c0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 shutdown_connections 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ecfe55f0 0x5620ecfe7a40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfe0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfd9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/3938821066 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfcfb30 0x5620ecfd99b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 >> 172.21.15.90:0/3938821066 conn(0x5620ecfcb1a0 msgr2=0x5620ecfca310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43d6a00640 0 client.26126 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 shutdown_connections 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.130+0000 7f43fa9b2740 1 -- 172.21.15.90:0/3938821066 wait complete. 2024-06-02T15:51:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.131+0000 7f43fa9b2740 20 client.26126 trim_cache size 0 max 16384 2024-06-02T15:51:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 Processor -- start 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 -- start start 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 0x5620ecfd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfe0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 0x5620ecfce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01b4550 con 0x5620ecfcfb30 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.158+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01b1d60 con 0x5620ecfd5690 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43f5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 0x5620ecfd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43f5c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 0x5620ecfce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43f5c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 0x5620ecfce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50802/0 (socket says 172.21.15.90:50802) 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43f5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 0x5620ecfd9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56712/0 (socket says 172.21.15.90:56712) 2024-06-02T15:51:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43f5c00640 1 -- 172.21.15.90:0/1189338905 learned_addr learned my addr 172.21.15.90:0/1189338905 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43efe00640 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfe0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01afbb0 con 0x5620ecfd0520 2024-06-02T15:51:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.159+0000 7f43efe00640 1 -- 172.21.15.90:0/1189338905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 msgr2=0x5620ecfce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43efe00640 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 0x5620ecfce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43efe00640 1 -- 172.21.15.90:0/1189338905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 msgr2=0x5620ecfd9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43efe00640 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 0x5620ecfd9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43efe00640 1 -- 172.21.15.90:0/1189338905 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ecfcbe70 con 0x5620ecfd0520 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43efe00640 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfe0ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43dc1001f0 tx=0x7f43dc03b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d98c2287984a248 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.160+0000 7f43e1600640 1 -- 172.21.15.90:0/1189338905 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc049040 con 0x5620ecfd0520 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.161+0000 7f43e1600640 1 -- 172.21.15.90:0/1189338905 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc0ca410 con 0x5620ecfd0520 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 msgr2=0x5620ecfe0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43e1600640 1 -- 172.21.15.90:0/1189338905 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc049040 con 0x5620ecfd0520 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfe0ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f43dc1001f0 tx=0x7f43dc03b720 comp rx=0 tx=0).stop 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 shutdown_connections 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd5690 0x5620ecfce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfd0520 0x5620ecfe0ef0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/1189338905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfcfb30 0x5620ecfd9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.162+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 >> 172.21.15.90:0/1189338905 conn(0x5620ecfcb1a0 msgr2=0x5620ecfce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.163+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 shutdown_connections 2024-06-02T15:51:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.163+0000 7f43fa9b2740 1 -- 172.21.15.90:0/1189338905 wait complete. 2024-06-02T15:51:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.163+0000 7f43fa9b2740 1 -- start start 2024-06-02T15:51:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.164+0000 7f43fa9b2740 1 Processor -- start 2024-06-02T15:51:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.165+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.165+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.165+0000 7f43fa9b2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43f5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43f5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44386/0 (socket says 172.21.15.90:44386) 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43f5c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43f5c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfdbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56724/0 (socket says 172.21.15.90:56724) 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43efe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50814/0 (socket says 172.21.15.90:50814) 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01b0220 con 0x5620ecfd5690 2024-06-02T15:51:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01b1690 con 0x5620ecfd0520 2024-06-02T15:51:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43fa9b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43f01b08f0 con 0x5620ecfcfb30 2024-06-02T15:51:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43fa9b2740 10 client.0 _ll_register_callbacks cb 0x7f43e40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.166+0000 7f43f5c00640 1 -- 172.21.15.90:0/4087144059 learned_addr learned my addr 172.21.15.90:0/4087144059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.167+0000 7f43efe00640 1 -- 172.21.15.90:0/4087144059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 msgr2=0x5620ecfdba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.167+0000 7f43efe00640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.167+0000 7f43efe00640 1 -- 172.21.15.90:0/4087144059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 msgr2=0x5620ecfdbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.167+0000 7f43f5200640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43efe00640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfdbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43efe00640 1 -- 172.21.15.90:0/4087144059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ecfd8820 con 0x5620ecfd0520 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43efe00640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5620ecfdab50 tx=0x7f43dc0c61d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513273e449e4c770 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0e9060 con 0x5620ecfd0520 2024-06-02T15:51:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.168+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f43dc0ca180 con 0x5620ecfd0520 2024-06-02T15:51:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.169+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc0ca410 con 0x5620ecfd0520 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.169+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f43dc049070 con 0x5620ecfd0520 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.169+0000 7f43fa9b2740 10 client.26132 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.169+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5620ed0603e0 con 0x5620ecfd0520 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.169+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc0ca410 con 0x5620ecfd0520 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f43dc0d26e0 con 0x5620ecfd0520 2024-06-02T15:51:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 10 client.26132 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 20 client.26132 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ed060a80 0x5620ed062ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 4 client.26132 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:51:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.170+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620ecf3b2c0 con 0x5620ed060a80 2024-06-02T15:51:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.173+0000 7f43f5200640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ed060a80 0x5620ed062ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.173+0000 7f43f5200640 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ed060a80 0x5620ed062ea0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578e93acc4c9976b server_cookie=a3babb6661e2129e in_seq=0 out_seq=0 2024-06-02T15:51:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.173+0000 7f43d6a00640 10 client.26132 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:23 smithi120 ceph-mon[36451]: pgmap v722: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 859 op/s 2024-06-02T15:51:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:23 smithi135 ceph-mon[36839]: pgmap v722: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 859 op/s 2024-06-02T15:51:23.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.581+0000 7f43d6a00640 1 -- 172.21.15.90:0/4087144059 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5620ecf3b2c0 con 0x5620ed060a80 2024-06-02T15:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.581+0000 7f43d6a00640 10 client.26132 handle_command_reply: tid=0 2024-06-02T15:51:23.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.581+0000 7f43fa9b2740 1 client.26132 shutdown 2024-06-02T15:51:23.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.582+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 msgr2=0x5620ecfe0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.582+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5620ecfdab50 tx=0x7f43dc0c61d0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 shutdown_connections 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5620ed060a80 0x5620ed062ea0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5620ecfd5690 0x5620ecfdbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5620ecfd0520 0x5620ecfe0f10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 --2- 172.21.15.90:0/4087144059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5620ecfcfb30 0x5620ecfdba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 >> 172.21.15.90:0/4087144059 conn(0x5620ecfcb1a0 msgr2=0x5620ecfd0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43d6a00640 0 client.26132 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.583+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 shutdown_connections 2024-06-02T15:51:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.584+0000 7f43fa9b2740 1 -- 172.21.15.90:0/4087144059 wait complete. 2024-06-02T15:51:23.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.584+0000 7f43fa9b2740 20 client.26132 trim_cache size 0 max 16384 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 msgr2=0x7f43d80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 0x7f43d80b4eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5620ecfcc070 tx=0x7f43e4009110 comp rx=0 tx=0).stop 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 msgr2=0x7f43f019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005940 tx=0x7f43dc004430 comp rx=0 tx=0).stop 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 shutdown_connections 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f43d80b29f0 0x7f43d80b4eb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f010a940 0x7f43f01a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43f0076a30 0x7f43f019b3e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 --2- 172.21.15.90:0/3487275322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43f0075450 0x7f43f019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:23.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.586+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 >> 172.21.15.90:0/3487275322 conn(0x7f43f00fe040 msgr2=0x7f43f01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.587+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 shutdown_connections 2024-06-02T15:51:23.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:23.587+0000 7f43f7c00640 1 -- 172.21.15.90:0/3487275322 wait complete. 2024-06-02T15:51:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:23 smithi090 ceph-mon[31595]: pgmap v722: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 859 op/s 2024-06-02T15:51:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:24 smithi120 ceph-mon[36451]: pgmap v723: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 614 op/s 2024-06-02T15:51:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:24 smithi135 ceph-mon[36839]: pgmap v723: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 614 op/s 2024-06-02T15:51:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:24 smithi090 ceph-mon[31595]: pgmap v723: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 614 op/s 2024-06-02T15:51:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:27 smithi120 ceph-mon[36451]: pgmap v724: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 827 op/s 2024-06-02T15:51:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:27 smithi135 ceph-mon[36839]: pgmap v724: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 827 op/s 2024-06-02T15:51:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:27 smithi090 ceph-mon[31595]: pgmap v724: 129 pgs: 129 active+clean; 159 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 827 op/s 2024-06-02T15:51:28.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:28 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:28.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:28 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:28.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:28 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:29.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:29 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:29.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:29 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:29.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:29 smithi090 ceph-mon[31595]: pgmap v725: 129 pgs: 129 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-02T15:51:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:29 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:29 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:29 smithi120 ceph-mon[36451]: pgmap v725: 129 pgs: 129 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-02T15:51:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:29 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:29 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:29 smithi135 ceph-mon[36839]: pgmap v725: 129 pgs: 129 active+clean; 162 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2024-06-02T15:51:30.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:30 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:30.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:30 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:30.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:30 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:31 smithi090 ceph-mon[31595]: pgmap v726: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 734 op/s 2024-06-02T15:51:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:31 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:31 smithi120 ceph-mon[36451]: pgmap v726: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 734 op/s 2024-06-02T15:51:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:31 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:31 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:31 smithi135 ceph-mon[36839]: pgmap v726: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 734 op/s 2024-06-02T15:51:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:31 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:31 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:33 smithi120 ceph-mon[36451]: pgmap v727: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 858 op/s 2024-06-02T15:51:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:51:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:51:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:33 smithi135 ceph-mon[36839]: pgmap v727: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 858 op/s 2024-06-02T15:51:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:51:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:51:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:33 smithi090 ceph-mon[31595]: pgmap v727: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 858 op/s 2024-06-02T15:51:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:51:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:51:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:51:34.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:51:34.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:51:34.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:51:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:35 smithi120 ceph-mon[36451]: pgmap v728: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 797 op/s 2024-06-02T15:51:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:35 smithi135 ceph-mon[36839]: pgmap v728: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 797 op/s 2024-06-02T15:51:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:35 smithi090 ceph-mon[31595]: pgmap v728: 129 pgs: 129 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 797 op/s 2024-06-02T15:51:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 -- 172.21.15.90:0/3235784063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 msgr2=0x7f7f3810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 --2- 172.21.15.90:0/3235784063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810a8f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009900 tx=0x7f7f2802a3f0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 -- 172.21.15.90:0/3235784063 shutdown_connections 2024-06-02T15:51:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 --2- 172.21.15.90:0/3235784063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f3810ae30 0x7f7f3810b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 --2- 172.21.15.90:0/3235784063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810a8f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 --2- 172.21.15.90:0/3235784063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f38109b30 0x7f7f38109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 -- 172.21.15.90:0/3235784063 >> 172.21.15.90:0/3235784063 conn(0x7f7f380709f0 msgr2=0x7f7f38072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 -- 172.21.15.90:0/3235784063 shutdown_connections 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.763+0000 7f7f3e400640 1 -- 172.21.15.90:0/3235784063 wait complete. 2024-06-02T15:51:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.764+0000 7f7f3e400640 1 Processor -- start 2024-06-02T15:51:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.765+0000 7f7f3e400640 1 -- start start 2024-06-02T15:51:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.765+0000 7f7f3e400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 0x7f7f3810e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.765+0000 7f7f3e400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.765+0000 7f7f3e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.765+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44320/0 (socket says 172.21.15.90:44320) 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f37400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 0x7f7f3810e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f37400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 0x7f7f3810e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:46224/0 (socket says 172.21.15.90:46224) 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50332/0 (socket says 172.21.15.90:50332) 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f3e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f38119060 con 0x7f7f3810ae30 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f3e400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f38118ee0 con 0x7f7f38109b30 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f36a00640 1 -- 172.21.15.90:0/2958081138 learned_addr learned my addr 172.21.15.90:0/2958081138 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.766+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381191e0 con 0x7f7f3810a490 2024-06-02T15:51:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.767+0000 7f7f37e00640 1 -- 172.21.15.90:0/2958081138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 msgr2=0x7f7f3810e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.767+0000 7f7f37e00640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.767+0000 7f7f37e00640 1 -- 172.21.15.90:0/2958081138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 msgr2=0x7f7f3810e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.767+0000 7f7f37e00640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 0x7f7f3810e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.767+0000 7f7f37e00640 1 -- 172.21.15.90:0/2958081138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f381b7f00 con 0x7f7f3810ae30 2024-06-02T15:51:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.768+0000 7f7f37e00640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000e960 tx=0x7f7f3000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.768+0000 7f7f2fe00640 1 -- 172.21.15.90:0/2958081138 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30004d20 con 0x7f7f3810ae30 2024-06-02T15:51:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.769+0000 7f7f2fe00640 1 -- 172.21.15.90:0/2958081138 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f30005020 con 0x7f7f3810ae30 2024-06-02T15:51:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.769+0000 7f7f2fe00640 1 -- 172.21.15.90:0/2958081138 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300116d0 con 0x7f7f3810ae30 2024-06-02T15:51:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.770+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f381b8190 con 0x7f7f3810ae30 2024-06-02T15:51:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.770+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f381b8660 con 0x7f7f3810ae30 2024-06-02T15:51:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.774+0000 7f7f411e1740 1 Processor -- start 2024-06-02T15:51:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.778+0000 7f7f2fe00640 1 -- 172.21.15.90:0/2958081138 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7f30022060 con 0x7f7f3810ae30 2024-06-02T15:51:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.778+0000 7f7f411e1740 1 -- start start 2024-06-02T15:51:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.779+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.779+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.779+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50334/0 (socket says 172.21.15.90:50334) 2024-06-02T15:51:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f36a00640 1 -- 172.21.15.90:0/4205114856 learned_addr learned my addr 172.21.15.90:0/4205114856 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be79462020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be79464b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be793ccb90 con 0x55be7945cb30 2024-06-02T15:51:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55be793bc730 con 0x55be79462690 2024-06-02T15:51:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.780+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55be793c0770 con 0x55be7945d520 2024-06-02T15:51:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f37400640 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be79462020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f36a00640 1 -- 172.21.15.90:0/4205114856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 msgr2=0x55be79462020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f36a00640 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be79462020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f36a00640 1 -- 172.21.15.90:0/4205114856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 msgr2=0x55be79464b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f36a00640 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be79464b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f36a00640 1 -- 172.21.15.90:0/4205114856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be793558b0 con 0x55be7945cb30 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.781+0000 7f7f2fe00640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 0x7f7f1c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.782+0000 7f7f36a00640 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7f28004160 tx=0x7f7f28004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f5dec1b41272f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.782+0000 7f7f2fe00640 1 -- 172.21.15.90:0/2958081138 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f30020070 con 0x7f7f3810ae30 2024-06-02T15:51:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.782+0000 7f7f37400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 0x7f7f1c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.783+0000 7f7f2cc00640 1 -- 172.21.15.90:0/4205114856 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28035030 con 0x55be7945cb30 2024-06-02T15:51:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.783+0000 7f7f2cc00640 1 -- 172.21.15.90:0/4205114856 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f280054f0 con 0x55be7945cb30 2024-06-02T15:51:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.783+0000 7f7f2cc00640 1 -- 172.21.15.90:0/4205114856 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28035030 con 0x55be7945cb30 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 msgr2=0x55be7945cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7f28004160 tx=0x7f7f28004190 comp rx=0 tx=0).stop 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f37400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 0x7f7f1c0b4de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55be7945d920 tx=0x7f7f2000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 shutdown_connections 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be79464b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be79462020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 --2- 172.21.15.90:0/4205114856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7945cf50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 >> 172.21.15.90:0/4205114856 conn(0x55be794581a0 msgr2=0x55be794585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.784+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 shutdown_connections 2024-06-02T15:51:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.785+0000 7f7f411e1740 1 -- 172.21.15.90:0/4205114856 wait complete. 2024-06-02T15:51:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.785+0000 7f7f411e1740 1 -- start start 2024-06-02T15:51:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 Processor -- start 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be79466e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be7946de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be790a2790 con 0x55be7945d520 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55be79364bc0 con 0x55be7945cb30 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55be794655e0 con 0x55be79462690 2024-06-02T15:51:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.786+0000 7f7f411e1740 10 client.0 _ll_register_callbacks cb 0x55be79342400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:46230/0 (socket says 172.21.15.90:46230) 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 -- 172.21.15.90:0/1315504384 learned_addr learned my addr 172.21.15.90:0/1315504384 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 -- 172.21.15.90:0/1315504384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 msgr2=0x55be7946de40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be7946de40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 -- 172.21.15.90:0/1315504384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 msgr2=0x55be79466e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be79466e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 -- 172.21.15.90:0/1315504384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be790d1210 con 0x55be7945cb30 2024-06-02T15:51:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f37400640 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f20003eb0 tx=0x7f7f200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184b3e5a06a86e96 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.787+0000 7f7f25600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20005220 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f280054f0 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f20005020 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f20005220 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f411e1740 10 client.26141 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.788+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55be7919d620 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f280054f0 con 0x55be7945cb30 2024-06-02T15:51:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7f20024070 con 0x55be7945cb30 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 10 client.26141 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 20 client.26141 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be79472510 0x55be794749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 4 client.26141 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.789+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be793ceae0 con 0x55be79472510 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.790+0000 7f7f36a00640 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be79472510 0x55be794749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.791+0000 7f7f25600640 10 client.26141 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.791+0000 7f7f36a00640 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be79472510 0x55be794749d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91dfcf7c31e0cf04 server_cookie=73b113778e2785f5 in_seq=0 out_seq=0 2024-06-02T15:51:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.792+0000 7f7f25600640 1 -- 172.21.15.90:0/1315504384 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be793ceae0 con 0x55be79472510 2024-06-02T15:51:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.792+0000 7f7f25600640 10 client.26141 handle_command_reply: tid=0 2024-06-02T15:51:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.792+0000 7f7f411e1740 1 client.26141 shutdown 2024-06-02T15:51:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.792+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 msgr2=0x55be794668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.792+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f20003eb0 tx=0x7f7f200046c0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 shutdown_connections 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be79472510 0x55be794749d0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be7946de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be79466e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 --2- 172.21.15.90:0/1315504384 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be794668c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 >> 172.21.15.90:0/1315504384 conn(0x55be794581a0 msgr2=0x55be79457310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f25600640 0 client.26141 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 shutdown_connections 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 1 -- 172.21.15.90:0/1315504384 wait complete. 2024-06-02T15:51:36.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.793+0000 7f7f411e1740 20 client.26141 trim_cache size 0 max 16384 2024-06-02T15:51:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 Processor -- start 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 -- start start 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be79456a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be7946de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c5590 con 0x55be7945d520 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c2da0 con 0x55be7945cb30 2024-06-02T15:51:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.823+0000 7f7f411e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c0bf0 con 0x55be79462690 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44356/0 (socket says 172.21.15.90:44356) 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 -- 172.21.15.90:0/880657423 learned_addr learned my addr 172.21.15.90:0/880657423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37400640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be79456a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f36a00640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be7946de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 -- 172.21.15.90:0/880657423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 msgr2=0x55be79456a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be79456a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 -- 172.21.15.90:0/880657423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 msgr2=0x55be7946de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.824+0000 7f7f37e00640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be7946de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f37e00640 1 -- 172.21.15.90:0/880657423 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be79475050 con 0x55be79462690 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f36a00640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be7946de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f37e00640 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f300a91c0 tx=0x7f7f300a9110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36fdc5e9c1ddaa8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f27e00640 1 -- 172.21.15.90:0/880657423 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300c3070 con 0x55be79462690 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f27e00640 1 -- 172.21.15.90:0/880657423 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f300ad5e0 con 0x55be79462690 2024-06-02T15:51:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.825+0000 7f7f27e00640 1 -- 172.21.15.90:0/880657423 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3001f030 con 0x55be79462690 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.826+0000 7f7f411e1740 1 -- 172.21.15.90:0/880657423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 msgr2=0x55be79456fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.826+0000 7f7f411e1740 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7f300a91c0 tx=0x7f7f300a9110 comp rx=0 tx=0).stop 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 -- 172.21.15.90:0/880657423 shutdown_connections 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be79462690 0x55be79456fc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945d520 0x55be7946de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 --2- 172.21.15.90:0/880657423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be7945cb30 0x55be79456a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 -- 172.21.15.90:0/880657423 >> 172.21.15.90:0/880657423 conn(0x55be794581a0 msgr2=0x55be7945b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 -- 172.21.15.90:0/880657423 shutdown_connections 2024-06-02T15:51:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 -- 172.21.15.90:0/880657423 wait complete. 2024-06-02T15:51:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.827+0000 7f7f411e1740 1 -- start start 2024-06-02T15:51:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 Processor -- start 2024-06-02T15:51:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7946b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be7946b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be7946c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c1260 con 0x55be7945cb30 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c26d0 con 0x55be79462690 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.828+0000 7f7f411e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f381c1930 con 0x55be7945d520 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f411e1740 10 client.0 _ll_register_callbacks cb 0x7f7f20004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be7946b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f36a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be7946b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44366/0 (socket says 172.21.15.90:44366) 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f36a00640 1 -- 172.21.15.90:0/201342368 learned_addr learned my addr 172.21.15.90:0/201342368 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f37e00640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be7946c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.829+0000 7f7f37400640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7946b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 -- 172.21.15.90:0/201342368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 msgr2=0x55be7946b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be7946b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 -- 172.21.15.90:0/201342368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 msgr2=0x55be7946b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7946b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 -- 172.21.15.90:0/201342368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be79465980 con 0x55be79462690 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f37e00640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be7946c6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f300a8f30 tx=0x7f7f30004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b6d7a945371ad4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.830+0000 7f7f25600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300c3070 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f7f300b1e10 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f300ad5e0 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f3001f040 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f411e1740 10 client.26147 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55be7946ddc0 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f300ad5e0 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7f3000f040 con 0x55be79462690 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f411e1740 10 client.26147 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.831+0000 7f7f411e1740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f411e1740 20 client.26147 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:36.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be794dd520 0x55be794df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f411e1740 4 client.26147 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:51:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be793c82c0 con 0x55be794dd520 2024-06-02T15:51:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f37400640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be794dd520 0x55be794df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f37400640 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be794dd520 0x55be794df940 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60c4432467403f7 server_cookie=b4d51bedd2e7a37a in_seq=0 out_seq=0 2024-06-02T15:51:36.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:36.832+0000 7f7f25600640 10 client.26147 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.482+0000 7f7f25600640 1 -- 172.21.15.90:0/201342368 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be793c82c0 con 0x55be794dd520 2024-06-02T15:51:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.482+0000 7f7f25600640 10 client.26147 handle_command_reply: tid=0 2024-06-02T15:51:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.482+0000 7f7f411e1740 1 client.26147 shutdown 2024-06-02T15:51:37.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.483+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 msgr2=0x55be7946c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.483+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be7946c6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f300a8f30 tx=0x7f7f30004ab0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.483+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 shutdown_connections 2024-06-02T15:51:37.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.483+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55be794dd520 0x55be794df940 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55be79462690 0x55be7946c6b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55be7945d520 0x55be7946b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 --2- 172.21.15.90:0/201342368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be7945cb30 0x55be7946b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 >> 172.21.15.90:0/201342368 conn(0x55be794581a0 msgr2=0x55be7945df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f25600640 0 client.26147 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 shutdown_connections 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 1 -- 172.21.15.90:0/201342368 wait complete. 2024-06-02T15:51:37.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.484+0000 7f7f411e1740 20 client.26147 trim_cache size 0 max 16384 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 msgr2=0x7f7f1c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 0x7f7f1c0b4de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55be7945d920 tx=0x7f7f2000b390 comp rx=0 tx=0).stop 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 msgr2=0x7f7f38115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000e960 tx=0x7f7f3000ee20 comp rx=0 tx=0).stop 2024-06-02T15:51:37.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 shutdown_connections 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7f1c0b2920 0x7f7f1c0b4de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f3810ae30 0x7f7f38115a60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f3810a490 0x7f7f3810e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 --2- 172.21.15.90:0/2958081138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f38109b30 0x7f7f3810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.485+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 >> 172.21.15.90:0/2958081138 conn(0x7f7f380709f0 msgr2=0x7f7f38071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.487+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 shutdown_connections 2024-06-02T15:51:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:37.487+0000 7f7f3e400640 1 -- 172.21.15.90:0/2958081138 wait complete. 2024-06-02T15:51:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:37 smithi120 ceph-mon[36451]: pgmap v729: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.07k op/s 2024-06-02T15:51:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:37 smithi135 ceph-mon[36839]: pgmap v729: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.07k op/s 2024-06-02T15:51:37.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:37 smithi090 ceph-mon[31595]: pgmap v729: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.07k op/s 2024-06-02T15:51:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:39 smithi120 ceph-mon[36451]: pgmap v730: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 885 op/s 2024-06-02T15:51:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:39 smithi135 ceph-mon[36839]: pgmap v730: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 885 op/s 2024-06-02T15:51:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:39 smithi090 ceph-mon[31595]: pgmap v730: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 885 op/s 2024-06-02T15:51:40.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:51:40.641 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:51:41.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:41 smithi120 ceph-mon[36451]: pgmap v731: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 889 op/s 2024-06-02T15:51:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:41 smithi135 ceph-mon[36839]: pgmap v731: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 889 op/s 2024-06-02T15:51:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:41 smithi090 ceph-mon[31595]: pgmap v731: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 889 op/s 2024-06-02T15:51:42.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:42 smithi090 ceph-mon[31595]: pgmap v732: 129 pgs: 129 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 864 op/s 2024-06-02T15:51:42.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:42 smithi120 ceph-mon[36451]: pgmap v732: 129 pgs: 129 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 864 op/s 2024-06-02T15:51:42.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:42 smithi135 ceph-mon[36839]: pgmap v732: 129 pgs: 129 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 864 op/s 2024-06-02T15:51:43.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:43 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:44.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:43 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:44.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:43 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:51:45.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:44 smithi120 ceph-mon[36451]: pgmap v733: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2024-06-02T15:51:45.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:44 smithi135 ceph-mon[36839]: pgmap v733: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2024-06-02T15:51:45.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:44 smithi090 ceph-mon[31595]: pgmap v733: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2024-06-02T15:51:47.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:47 smithi090 ceph-mon[31595]: pgmap v734: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 810 op/s 2024-06-02T15:51:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:47 smithi120 ceph-mon[36451]: pgmap v734: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 810 op/s 2024-06-02T15:51:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:47 smithi135 ceph-mon[36839]: pgmap v734: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 810 op/s 2024-06-02T15:51:47.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 -- 172.21.15.90:0/3666946081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06bbe0 msgr2=0x7fccac1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3666946081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06bbe0 0x7fccac1103d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009f90 tx=0x7fcca400b330 comp rx=0 tx=0).stop 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 -- 172.21.15.90:0/3666946081 shutdown_connections 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3666946081 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06bbe0 0x7fccac1103d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3666946081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 0x7fccac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3666946081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccac06b220 0x7fccac06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.670+0000 7fccb3c00700 1 -- 172.21.15.90:0/3666946081 >> 172.21.15.90:0/3666946081 conn(0x7fccac0793f0 msgr2=0x7fccac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.671+0000 7fccb3c00700 1 -- 172.21.15.90:0/3666946081 shutdown_connections 2024-06-02T15:51:47.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.672+0000 7fccb3c00700 1 -- 172.21.15.90:0/3666946081 wait complete. 2024-06-02T15:51:47.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.673+0000 7fccb3c00700 1 Processor -- start 2024-06-02T15:51:47.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.673+0000 7fccb3c00700 1 -- start start 2024-06-02T15:51:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccac06bbe0 0x7fccac19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 0x7fccac1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1136c0 con 0x7fccac06bbe0 2024-06-02T15:51:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccac113540 con 0x7fccac06b220 2024-06-02T15:51:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.674+0000 7fccb3c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fccac113840 con 0x7fccac1095b0 2024-06-02T15:51:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42000/0 (socket says 172.21.15.90:42000) 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 -- 172.21.15.90:0/3734481472 learned_addr learned my addr 172.21.15.90:0/3734481472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 0x7fccac1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 -- 172.21.15.90:0/3734481472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 msgr2=0x7fccac1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 0x7fccac1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 -- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccac06bbe0 msgr2=0x7fccac19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccac06bbe0 0x7fccac19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.675+0000 7fccb1200700 1 -- 172.21.15.90:0/3734481472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac19a100 con 0x7fccac06b220 2024-06-02T15:51:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.676+0000 7fccb1200700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9a0 tx=0x7fcca000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.676+0000 7fcca9600700 1 -- 172.21.15.90:0/3734481472 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000cbd0 con 0x7fccac06b220 2024-06-02T15:51:47.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.676+0000 7fcca9600700 1 -- 172.21.15.90:0/3734481472 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca000cd70 con 0x7fccac06b220 2024-06-02T15:51:47.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.676+0000 7fcca9600700 1 -- 172.21.15.90:0/3734481472 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0018680 con 0x7fccac06b220 2024-06-02T15:51:47.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.676+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac19a3f0 con 0x7fccac06b220 2024-06-02T15:51:47.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.677+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac102440 con 0x7fccac06b220 2024-06-02T15:51:47.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.683+0000 7fcca9600700 1 -- 172.21.15.90:0/3734481472 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcca0018820 con 0x7fccac06b220 2024-06-02T15:51:47.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.683+0000 7fccbe5f8b80 1 Processor -- start 2024-06-02T15:51:47.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.683+0000 7fccbe5f8b80 1 -- start start 2024-06-02T15:51:47.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf4e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf48260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5588edea6e40 con 0x5588edf4e7d0 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5588ede969e0 con 0x5588edf48880 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.684+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5588ede9aa20 con 0x5588edf4deb0 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.685+0000 7fccabe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.685+0000 7fccabe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42010/0 (socket says 172.21.15.90:42010) 2024-06-02T15:51:47.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.685+0000 7fccabe00700 1 -- 172.21.15.90:0/2807374903 learned_addr learned my addr 172.21.15.90:0/2807374903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.685+0000 7fcca9600700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 0x7fcc940b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.685+0000 7fcca9600700 1 -- 172.21.15.90:0/3734481472 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcca00e2810 con 0x7fccac06b220 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.686+0000 7fccabe00700 1 -- 172.21.15.90:0/2807374903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 msgr2=0x5588edf4e290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.686+0000 7fccabe00700 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf4e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.686+0000 7fccabe00700 1 -- 172.21.15.90:0/2807374903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 msgr2=0x5588edf48260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.686+0000 7fccabe00700 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf48260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.686+0000 7fccabe00700 1 -- 172.21.15.90:0/2807374903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ede5f420 con 0x5588edf48880 2024-06-02T15:51:47.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.687+0000 7fccabe00700 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc980046e0 tx=0x7fcc98009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d5a7398254e8e79 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.688+0000 7fcc9e000700 1 -- 172.21.15.90:0/2807374903 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800eb60 con 0x5588edf48880 2024-06-02T15:51:47.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.688+0000 7fcc9e000700 1 -- 172.21.15.90:0/2807374903 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc98004ce0 con 0x5588edf48880 2024-06-02T15:51:47.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.688+0000 7fcc9e000700 1 -- 172.21.15.90:0/2807374903 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98010430 con 0x5588edf48880 2024-06-02T15:51:47.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.688+0000 7fccb1c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 0x7fcc940b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2807374903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 msgr2=0x5588edf48ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc980046e0 tx=0x7fcc98009ea0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccb1c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 0x7fcc940b4e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009790 tx=0x7fcca40096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2807374903 shutdown_connections 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf48880 0x5588edf48ce0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc980046e0 tx=0x7fcc98009ea0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf4e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2807374903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf48260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.689+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2807374903 >> 172.21.15.90:0/2807374903 conn(0x5588ede76be0 msgr2=0x5588edf43010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.690+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2807374903 shutdown_connections 2024-06-02T15:51:47.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.690+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2807374903 wait complete. 2024-06-02T15:51:47.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.691+0000 7fccbe5f8b80 1 -- start start 2024-06-02T15:51:47.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.691+0000 7fccbe5f8b80 1 Processor -- start 2024-06-02T15:51:47.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf52180 0x5588edf52600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5588edd9cfa0 con 0x5588edf52180 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5588ede37760 con 0x5588edf4e7d0 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5588edf55240 con 0x5588edf4deb0 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccbe5f8b80 10 client.0 _ll_register_callbacks cb 0x5588ede40c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.693+0000 7fccb1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf54860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42014/0 (socket says 172.21.15.90:42014) 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 -- 172.21.15.90:0/3848090152 learned_addr learned my addr 172.21.15.90:0/3848090152 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 -- 172.21.15.90:0/3848090152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 msgr2=0x5588edf54860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf54860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 -- 172.21.15.90:0/3848090152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf52180 msgr2=0x5588edf52600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf52180 0x5588edf52600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.694+0000 7fccabe00700 1 -- 172.21.15.90:0/3848090152 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588edde68d0 con 0x5588edf4e7d0 2024-06-02T15:51:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.695+0000 7fccabe00700 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc98000c00 tx=0x7fcc98005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4de604c9391c254 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.696+0000 7fcc92a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:51:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.696+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9801d040 con 0x5588edf4e7d0 2024-06-02T15:51:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.696+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcc9800e960 con 0x5588edf4e7d0 2024-06-02T15:51:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.697+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc98004ce0 con 0x5588edf4e7d0 2024-06-02T15:51:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.698+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcc98004b30 con 0x5588edf4e7d0 2024-06-02T15:51:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.699+0000 7fccbe5f8b80 10 client.26159 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.699+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5588ede67060 con 0x5588edf4e7d0 2024-06-02T15:51:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.699+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc98004ce0 con 0x5588edf4e7d0 2024-06-02T15:51:47.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.700+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcc98018bc0 con 0x5588edf4e7d0 2024-06-02T15:51:47.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.700+0000 7fccbe5f8b80 10 client.26159 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:47.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.700+0000 7fccbe5f8b80 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.700+0000 7fccbe5f8b80 20 client.26159 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.700+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edf6ab50 0x5588edf6cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.701+0000 7fccbe5f8b80 4 client.26159 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.701+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588edea8d90 con 0x5588edf6ab50 2024-06-02T15:51:47.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.701+0000 7fccb1200700 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edf6ab50 0x5588edf6cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.703+0000 7fccb1200700 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edf6ab50 0x5588edf6cf40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541d5455d2a3bfbd server_cookie=7a0c99f3ba6e21f8 in_seq=0 out_seq=0 2024-06-02T15:51:47.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.703+0000 7fcc92a00700 10 client.26159 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:47.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.704+0000 7fcc92a00700 1 -- 172.21.15.90:0/3848090152 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5588edea8d90 con 0x5588edf6ab50 2024-06-02T15:51:47.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.704+0000 7fcc92a00700 10 client.26159 handle_command_reply: tid=0 2024-06-02T15:51:47.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.704+0000 7fccbe5f8b80 1 client.26159 shutdown 2024-06-02T15:51:47.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.705+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 msgr2=0x5588edf51b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.705+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc98000c00 tx=0x7fcc98005da0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 shutdown_connections 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edf6ab50 0x5588edf6cf40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf52180 0x5588edf52600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/3848090152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf51b30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 >> 172.21.15.90:0/3848090152 conn(0x5588ede76be0 msgr2=0x5588edf47250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.706+0000 7fcc92a00700 0 client.26159 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:47.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.707+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 shutdown_connections 2024-06-02T15:51:47.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.707+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/3848090152 wait complete. 2024-06-02T15:51:47.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.708+0000 7fccbe5f8b80 20 client.26159 trim_cache size 0 max 16384 2024-06-02T15:51:47.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.740+0000 7fccbe5f8b80 1 Processor -- start 2024-06-02T15:51:47.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.740+0000 7fccbe5f8b80 1 -- start start 2024-06-02T15:51:47.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.740+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4deb0 0x5588edf48260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1afb80 con 0x5588edf4deb0 2024-06-02T15:51:47.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1ad350 con 0x5588edf4e7d0 2024-06-02T15:51:47.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccbe5f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1ab1a0 con 0x5588edf52180 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccb1c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccb1c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52130/0 (socket says 172.21.15.90:52130) 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.741+0000 7fccb1c00700 1 -- 172.21.15.90:0/2298302540 learned_addr learned my addr 172.21.15.90:0/2298302540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.742+0000 7fccb1c00700 1 -- 172.21.15.90:0/2298302540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 msgr2=0x5588edf54860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.742+0000 7fccb1c00700 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf54860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.742+0000 7fccb1c00700 1 -- 172.21.15.90:0/2298302540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4deb0 msgr2=0x5588edf48260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.742+0000 7fccb1c00700 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4deb0 0x5588edf48260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.742+0000 7fccb1c00700 1 -- 172.21.15.90:0/2298302540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588edf49020 con 0x5588edf52180 2024-06-02T15:51:47.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.743+0000 7fccb1c00700 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcca4004cc0 tx=0x7fcca400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7b5d72507bf1b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.743+0000 7fcc9d600700 1 -- 172.21.15.90:0/2298302540 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca40075e0 con 0x5588edf52180 2024-06-02T15:51:47.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.744+0000 7fcc9d600700 1 -- 172.21.15.90:0/2298302540 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca402e070 con 0x5588edf52180 2024-06-02T15:51:47.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.744+0000 7fcc9d600700 1 -- 172.21.15.90:0/2298302540 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca40299d0 con 0x5588edf52180 2024-06-02T15:51:47.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2298302540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 msgr2=0x5588edf44700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcca4004cc0 tx=0x7fcca400b330 comp rx=0 tx=0).stop 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2298302540 shutdown_connections 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf52180 0x5588edf44700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4deb0 0x5588edf48260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/2298302540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf4e7d0 0x5588edf54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.745+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2298302540 >> 172.21.15.90:0/2298302540 conn(0x5588ede76be0 msgr2=0x5588edf3ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.746+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2298302540 shutdown_connections 2024-06-02T15:51:47.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.746+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/2298302540 wait complete. 2024-06-02T15:51:47.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.747+0000 7fccbe5f8b80 1 -- start start 2024-06-02T15:51:47.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.747+0000 7fccbe5f8b80 1 Processor -- start 2024-06-02T15:51:47.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.748+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.748+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.748+0000 7fccbe5f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.749+0000 7fccabe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.749+0000 7fccabe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59578/0 (socket says 172.21.15.90:59578) 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.749+0000 7fccb1c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.749+0000 7fccb1c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42032/0 (socket says 172.21.15.90:42032) 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.749+0000 7fccb1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccb1200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52140/0 (socket says 172.21.15.90:52140) 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccabe00700 1 -- 172.21.15.90:0/4233865535 learned_addr learned my addr 172.21.15.90:0/4233865535 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1ab810 con 0x5588edf4e7d0 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1acc80 con 0x5588edf52180 2024-06-02T15:51:47.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fccac1abee0 con 0x5588edf4deb0 2024-06-02T15:51:47.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.750+0000 7fccbe5f8b80 10 client.0 _ll_register_callbacks cb 0x5588ede40c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:47.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.751+0000 7fccb1200700 1 -- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 msgr2=0x5588edf72d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.751+0000 7fccb1200700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.751+0000 7fccb1200700 1 -- 172.21.15.90:0/4233865535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 msgr2=0x5588edf72820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccb1200700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccb1200700 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588edfd46b0 con 0x5588edf4deb0 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca402e070 con 0x5588edf4deb0 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccb1c00700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccabe00700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:51:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fccb1200700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e970 tx=0x7fcca00a5a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d37c053f98b0117 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fcc92a00700 10 client.26086 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.752+0000 7fcc92a00700 1 -- 172.21.15.90:0/4233865535 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00a9590 con 0x5588edf4deb0 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fcc92a00700 1 -- 172.21.15.90:0/4233865535 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fcca00a9ef0 con 0x5588edf4deb0 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fcc92a00700 1 -- 172.21.15.90:0/4233865535 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcca00aa180 con 0x5588edf4deb0 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 10 client.26086 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5588edfd58e0 con 0x5588edf4deb0 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fcc92a00700 1 -- 172.21.15.90:0/4233865535 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcca00b2450 con 0x5588edf4deb0 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 20 client.26086 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edfd5f90 0x5588edfd83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 4 client.26086 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.753+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588edea2570 con 0x5588edfd5f90 2024-06-02T15:51:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.754+0000 7fccabe00700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edfd5f90 0x5588edfd83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:47.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.754+0000 7fccabe00700 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edfd5f90 0x5588edfd83b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233d20867750e1d3 server_cookie=c78e363974de764e in_seq=0 out_seq=0 2024-06-02T15:51:47.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.754+0000 7fcc92a00700 10 client.26086 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:47.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fcc92a00700 1 -- 172.21.15.90:0/4233865535 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5588edea2570 con 0x5588edfd5f90 2024-06-02T15:51:47.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fcc92a00700 10 client.26086 handle_command_reply: tid=0 2024-06-02T15:51:47.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 client.26086 shutdown 2024-06-02T15:51:47.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 msgr2=0x5588edf722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e970 tx=0x7fcca00a5a70 comp rx=0 tx=0).stop 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 shutdown_connections 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5588edfd5f90 0x5588edfd83b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5588edf52180 0x5588edf72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588edf4deb0 0x5588edf722e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 --2- 172.21.15.90:0/4233865535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588edf4e7d0 0x5588edf72820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 >> 172.21.15.90:0/4233865535 conn(0x5588ede76be0 msgr2=0x5588edf44700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.756+0000 7fcc92a00700 0 client.26086 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:47.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.757+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 shutdown_connections 2024-06-02T15:51:47.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.757+0000 7fccbe5f8b80 1 -- 172.21.15.90:0/4233865535 wait complete. 2024-06-02T15:51:47.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.757+0000 7fccbe5f8b80 20 client.26086 trim_cache size 0 max 16384 2024-06-02T15:51:47.755 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:51:47.755 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: "5bba5401-7f43-4eff-9382-22abf56b09a5": { 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "5bba5401-7f43-4eff-9382-22abf56b09a5", 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:51:47.756 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.758+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 msgr2=0x7fcc940b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.758+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 0x7fcc940b4e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009790 tx=0x7fcca40096c0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.758+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 msgr2=0x7fccac19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.758+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9a0 tx=0x7fcca000ed60 comp rx=0 tx=0).stop 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 shutdown_connections 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcc940b29d0 0x7fcc940b4e90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fccac1095b0 0x7fccac1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccac06b220 0x7fccac19f7e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 --2- 172.21.15.90:0/3734481472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccac06bbe0 0x7fccac19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:47.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 >> 172.21.15.90:0/3734481472 conn(0x7fccac0793f0 msgr2=0x7fccac101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:47.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 shutdown_connections 2024-06-02T15:51:47.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:51:47.759+0000 7fccb3c00700 1 -- 172.21.15.90:0/3734481472 wait complete. 2024-06-02T15:51:49.078 INFO:tasks.cephfs.filesystem:scrub status for tag:5bba5401-7f43-4eff-9382-22abf56b09a5 - {'path': '/', 'tag': '5bba5401-7f43-4eff-9382-22abf56b09a5', 'options': 'recursive,force'} 2024-06-02T15:51:49.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:49 smithi090 ceph-mon[31595]: pgmap v735: 129 pgs: 129 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-06-02T15:51:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:49 smithi120 ceph-mon[36451]: pgmap v735: 129 pgs: 129 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-06-02T15:51:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:49 smithi135 ceph-mon[36839]: pgmap v735: 129 pgs: 129 active+clean; 190 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 -- 172.21.15.90:0/2243197093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8241028e0 msgr2=0x7fb824102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 --2- 172.21.15.90:0/2243197093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8241028e0 0x7fb824102d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb818009900 tx=0x7fb81802a410 comp rx=0 tx=0).stop 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 -- 172.21.15.90:0/2243197093 shutdown_connections 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 --2- 172.21.15.90:0/2243197093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb824104490 0x7fb82410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 --2- 172.21.15.90:0/2243197093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 0x7fb824103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 --2- 172.21.15.90:0/2243197093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8241028e0 0x7fb824102d00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.692+0000 7fb829e00640 1 -- 172.21.15.90:0/2243197093 >> 172.21.15.90:0/2243197093 conn(0x7fb8240fde50 msgr2=0x7fb824100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.693+0000 7fb829e00640 1 -- 172.21.15.90:0/2243197093 shutdown_connections 2024-06-02T15:51:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.693+0000 7fb829e00640 1 -- 172.21.15.90:0/2243197093 wait complete. 2024-06-02T15:51:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.695+0000 7fb829e00640 1 Processor -- start 2024-06-02T15:51:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.695+0000 7fb829e00640 1 -- start start 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 0x7fb82419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 0x7fb8241a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410e400 con 0x7fb824104490 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410e280 con 0x7fb8241028e0 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb829e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410e580 con 0x7fb824103ad0 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb828a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb828a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42050/0 (socket says 172.21.15.90:42050) 2024-06-02T15:51:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb828a00640 1 -- 172.21.15.90:0/4110261496 learned_addr learned my addr 172.21.15.90:0/4110261496 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.696+0000 7fb823e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 0x7fb82419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb829400640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 0x7fb8241a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 -- 172.21.15.90:0/4110261496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 msgr2=0x7fb82419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 0x7fb82419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 -- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 msgr2=0x7fb8241a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 0x7fb8241a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 -- 172.21.15.90:0/4110261496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8241a2970 con 0x7fb8241028e0 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb829400640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 0x7fb8241a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.697+0000 7fb828a00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb818006010 tx=0x7fb8180042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.698+0000 7fb821600640 1 -- 172.21.15.90:0/4110261496 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81803f030 con 0x7fb8241028e0 2024-06-02T15:51:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.698+0000 7fb821600640 1 -- 172.21.15.90:0/4110261496 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8180318f0 con 0x7fb8241028e0 2024-06-02T15:51:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.698+0000 7fb821600640 1 -- 172.21.15.90:0/4110261496 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818035070 con 0x7fb8241028e0 2024-06-02T15:51:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.698+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8241a2c00 con 0x7fb8241028e0 2024-06-02T15:51:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.698+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a30a0 con 0x7fb8241028e0 2024-06-02T15:51:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.703+0000 7fb82e39b740 1 Processor -- start 2024-06-02T15:51:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.703+0000 7fb821600640 1 -- 172.21.15.90:0/4110261496 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb818042030 con 0x7fb8241028e0 2024-06-02T15:51:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.703+0000 7fb82e39b740 1 -- start start 2024-06-02T15:51:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb7850 0x561cf8bb9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cf8b21b90 con 0x561cf8bb7850 2024-06-02T15:51:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561cf8b11730 con 0x561cf8bb26e0 2024-06-02T15:51:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb82e39b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561cf8b15770 con 0x561cf8bb1cf0 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb829400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb829400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52172/0 (socket says 172.21.15.90:52172) 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb829400640 1 -- 172.21.15.90:0/695759636 learned_addr learned my addr 172.21.15.90:0/695759636 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.704+0000 7fb828a00640 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.705+0000 7fb829400640 1 -- 172.21.15.90:0/695759636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 msgr2=0x561cf8bb71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.705+0000 7fb829400640 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bb71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.705+0000 7fb829400640 1 -- 172.21.15.90:0/695759636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb7850 msgr2=0x561cf8bb9ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.705+0000 7fb829400640 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb7850 0x561cf8bb9ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.705+0000 7fb829400640 1 -- 172.21.15.90:0/695759636 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf8aaa8b0 con 0x561cf8bb1cf0 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.706+0000 7fb829400640 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f10 tx=0x7fb81c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e81fca355dc194d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.706+0000 7fb815c00640 1 -- 172.21.15.90:0/695759636 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c016070 con 0x561cf8bb1cf0 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.706+0000 7fb815c00640 1 -- 172.21.15.90:0/695759636 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c004d20 con 0x561cf8bb1cf0 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.706+0000 7fb815c00640 1 -- 172.21.15.90:0/695759636 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0117a0 con 0x561cf8bb1cf0 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb821600640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 0x7fb80c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 -- 172.21.15.90:0/695759636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 msgr2=0x561cf8bb2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009f10 tx=0x7fb81c00e6c0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 -- 172.21.15.90:0/695759636 shutdown_connections 2024-06-02T15:51:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb7850 0x561cf8bb9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 --2- 172.21.15.90:0/695759636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb2110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb82e39b740 1 -- 172.21.15.90:0/695759636 >> 172.21.15.90:0/695759636 conn(0x561cf8bad360 msgr2=0x561cf8bad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.707+0000 7fb821600640 1 -- 172.21.15.90:0/4110261496 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb818102aa0 con 0x7fb8241028e0 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.708+0000 7fb82e39b740 1 -- 172.21.15.90:0/695759636 shutdown_connections 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.708+0000 7fb828a00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 0x7fb80c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.708+0000 7fb82e39b740 1 -- 172.21.15.90:0/695759636 wait complete. 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.708+0000 7fb82e39b740 1 -- start start 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.708+0000 7fb828a00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 0x7fb80c0b4d20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb818100560 tx=0x7fb818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 Processor -- start 2024-06-02T15:51:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb1cf0 0x561cf8bbbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb26e0 0x561cf8bbc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cf87f7790 con 0x561cf8bb26e0 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561cf8ab9bc0 con 0x561cf8bb1cf0 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561cf8bba800 con 0x561cf8bb7850 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.709+0000 7fb82e39b740 10 client.0 _ll_register_callbacks cb 0x561cf8a97400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.710+0000 7fb823e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb26e0 0x561cf8bbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.710+0000 7fb829400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.710+0000 7fb829400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52186/0 (socket says 172.21.15.90:52186) 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.710+0000 7fb829400640 1 -- 172.21.15.90:0/374679456 learned_addr learned my addr 172.21.15.90:0/374679456 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.710+0000 7fb829400640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb1cf0 0x561cf8bbbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 -- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb1cf0 msgr2=0x561cf8bbbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb1cf0 0x561cf8bbbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 -- 172.21.15.90:0/374679456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb26e0 msgr2=0x561cf8bbc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb26e0 0x561cf8bbc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 -- 172.21.15.90:0/374679456 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf8826210 con 0x561cf8bb7850 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb829400640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ca0 tx=0x7fb81c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b2e5349d0377e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb80a000640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c016070 con 0x561cf8bb7850 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb81c004ed0 con 0x561cf8bb7850 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.711+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c004d20 con 0x561cf8bb7850 2024-06-02T15:51:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb81c032050 con 0x561cf8bb7850 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 10 client.26098 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561cf88f2620 con 0x561cf8bb7850 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c005580 con 0x561cf8bb7850 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb81c00f040 con 0x561cf8bb7850 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 10 client.26098 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 20 client.26098 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.712+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8bc72b0 0x561cf8bc96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.713+0000 7fb82e39b740 4 client.26098 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:51:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.713+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb81c004d20 con 0x561cf8bc72b0 2024-06-02T15:51:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.808+0000 7fb823e00640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8bc72b0 0x561cf8bc96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.809+0000 7fb80a000640 10 client.26098 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.809+0000 7fb823e00640 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8bc72b0 0x561cf8bc96a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b56401ce9f5941 server_cookie=c1747c6f075c1c03 in_seq=0 out_seq=0 2024-06-02T15:51:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb80a000640 1 -- 172.21.15.90:0/374679456 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fb81c004d20 con 0x561cf8bc72b0 2024-06-02T15:51:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb80a000640 10 client.26098 handle_command_reply: tid=0 2024-06-02T15:51:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 client.26098 shutdown 2024-06-02T15:51:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 msgr2=0x561cf8bc3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ca0 tx=0x7fb81c00eef0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 shutdown_connections 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8bc72b0 0x561cf8bc96a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc3040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb26e0 0x561cf8bbc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 --2- 172.21.15.90:0/374679456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb1cf0 0x561cf8bbbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 >> 172.21.15.90:0/374679456 conn(0x561cf8bad360 msgr2=0x561cf8bac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb80a000640 0 client.26098 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.816+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 shutdown_connections 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.817+0000 7fb82e39b740 1 -- 172.21.15.90:0/374679456 wait complete. 2024-06-02T15:51:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.817+0000 7fb82e39b740 20 client.26098 trim_cache size 0 max 16384 2024-06-02T15:51:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.845+0000 7fb82e39b740 1 Processor -- start 2024-06-02T15:51:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.845+0000 7fb82e39b740 1 -- start start 2024-06-02T15:51:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb26e0 0x561cf8bc3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb7850 0x561cf8bb3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241b0080 con 0x561cf8bb1cf0 2024-06-02T15:51:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ad890 con 0x561cf8bb7850 2024-06-02T15:51:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb82e39b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ab6e0 con 0x561cf8bb26e0 2024-06-02T15:51:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb829400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb829400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59612/0 (socket says 172.21.15.90:59612) 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb829400640 1 -- 172.21.15.90:0/1337194988 learned_addr learned my addr 172.21.15.90:0/1337194988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.846+0000 7fb829400640 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb7850 0x561cf8bb3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb823e00640 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb26e0 0x561cf8bc3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 -- 172.21.15.90:0/1337194988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb26e0 msgr2=0x561cf8bc3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb26e0 0x561cf8bc3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 -- 172.21.15.90:0/1337194988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb7850 msgr2=0x561cf8bb3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb7850 0x561cf8bb3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 -- 172.21.15.90:0/1337194988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf8bc9cd0 con 0x561cf8bb1cf0 2024-06-02T15:51:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.847+0000 7fb829400640 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb81c010010 tx=0x7fb81c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6c0c6df799a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.848+0000 7fb815200640 1 -- 172.21.15.90:0/1337194988 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c016070 con 0x561cf8bb1cf0 2024-06-02T15:51:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.848+0000 7fb815200640 1 -- 172.21.15.90:0/1337194988 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c003e20 con 0x561cf8bb1cf0 2024-06-02T15:51:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.848+0000 7fb815200640 1 -- 172.21.15.90:0/1337194988 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c02f040 con 0x561cf8bb1cf0 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 -- 172.21.15.90:0/1337194988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 msgr2=0x561cf8bb0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb81c010010 tx=0x7fb81c00e6c0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 -- 172.21.15.90:0/1337194988 shutdown_connections 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb7850 0x561cf8bb3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb26e0 0x561cf8bc3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 --2- 172.21.15.90:0/1337194988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bb0980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 -- 172.21.15.90:0/1337194988 >> 172.21.15.90:0/1337194988 conn(0x561cf8bad360 msgr2=0x561cf8bb0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 -- 172.21.15.90:0/1337194988 shutdown_connections 2024-06-02T15:51:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.849+0000 7fb82e39b740 1 -- 172.21.15.90:0/1337194988 wait complete. 2024-06-02T15:51:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.850+0000 7fb82e39b740 1 -- start start 2024-06-02T15:51:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.850+0000 7fb82e39b740 1 Processor -- start 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241abd50 con 0x561cf8bb1cf0 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ad1c0 con 0x561cf8bb26e0 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241ac420 con 0x561cf8bb7850 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb82e39b740 10 client.0 _ll_register_callbacks cb 0x7fb81c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb829400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb829400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59628/0 (socket says 172.21.15.90:59628) 2024-06-02T15:51:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.851+0000 7fb829400640 1 -- 172.21.15.90:0/3126921602 learned_addr learned my addr 172.21.15.90:0/3126921602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:51:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb823e00640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 -- 172.21.15.90:0/3126921602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 msgr2=0x561cf8bc1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 -- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 msgr2=0x561cf8bc0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bc0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 -- 172.21.15.90:0/3126921602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf8bbaba0 con 0x561cf8bb1cf0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb823e00640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bc0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb829400640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000f80 tx=0x7fb81c011470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11307dfec87e3bff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.852+0000 7fb80a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c016030 con 0x561cf8bb1cf0 2024-06-02T15:51:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fb81c004000 con 0x561cf8bb1cf0 2024-06-02T15:51:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb81c003e20 con 0x561cf8bb1cf0 2024-06-02T15:51:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb81c047050 con 0x561cf8bb1cf0 2024-06-02T15:51:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb82e39b740 10 client.15930 fetch_fsmap learned FSMap version 37 2024-06-02T15:51:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.853+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561cf8bad190 con 0x561cf8bb1cf0 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c003e20 con 0x561cf8bb1cf0 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb81c029690 con 0x561cf8bb1cf0 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb82e39b740 10 client.15930 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb82e39b740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb82e39b740 20 client.15930 populate_metadata read hostname 'smithi090' 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.854+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8c3abc0 0x561cf8c3cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.855+0000 7fb82e39b740 4 client.15930 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.855+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cf8b23ae0 con 0x561cf8c3abc0 2024-06-02T15:51:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.855+0000 7fb823e00640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8c3abc0 0x561cf8c3cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:51:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.855+0000 7fb823e00640 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8c3abc0 0x561cf8c3cfe0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa12d66bed55478 server_cookie=8874656e018ef34f in_seq=0 out_seq=0 2024-06-02T15:51:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:50.855+0000 7fb80a000640 10 client.15930 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:51 smithi120 ceph-mon[36451]: pgmap v736: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 549 op/s 2024-06-02T15:51:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:51 smithi135 ceph-mon[36839]: pgmap v736: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 549 op/s 2024-06-02T15:51:51.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:51 smithi090 ceph-mon[31595]: pgmap v736: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 549 op/s 2024-06-02T15:51:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb80a000640 1 -- 172.21.15.90:0/3126921602 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561cf8b23ae0 con 0x561cf8c3abc0 2024-06-02T15:51:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb80a000640 10 client.15930 handle_command_reply: tid=0 2024-06-02T15:51:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb82e39b740 1 client.15930 shutdown 2024-06-02T15:51:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 msgr2=0x561cf8bc0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000f80 tx=0x7fb81c011470 comp rx=0 tx=0).stop 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.325+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 shutdown_connections 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x561cf8c3abc0 0x561cf8c3cfe0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cf8bb7850 0x561cf8bc1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561cf8bb26e0 0x561cf8bc0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 --2- 172.21.15.90:0/3126921602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8bb1cf0 0x561cf8bc0750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 >> 172.21.15.90:0/3126921602 conn(0x561cf8bad360 msgr2=0x561cf8bb70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:52.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb80a000640 0 client.15930 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:51:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.326+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 shutdown_connections 2024-06-02T15:51:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.327+0000 7fb82e39b740 1 -- 172.21.15.90:0/3126921602 wait complete. 2024-06-02T15:51:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.327+0000 7fb82e39b740 20 client.15930 trim_cache size 0 max 16384 2024-06-02T15:51:52.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:51:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:51:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:51:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.328+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 msgr2=0x7fb80c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.328+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 0x7fb80c0b4d20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb818100560 tx=0x7fb818009260 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.328+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 msgr2=0x7fb82419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.328+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb818006010 tx=0x7fb8180042f0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 shutdown_connections 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb80c0b2860 0x7fb80c0b4d20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb824104490 0x7fb8241a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb824103ad0 0x7fb82419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 --2- 172.21.15.90:0/4110261496 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb8241028e0 0x7fb82419ada0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:51:52.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 >> 172.21.15.90:0/4110261496 conn(0x7fb8240fde50 msgr2=0x7fb8241092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:51:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.329+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 shutdown_connections 2024-06-02T15:51:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:51:52.330+0000 7fb829e00640 1 -- 172.21.15.90:0/4110261496 wait complete. 2024-06-02T15:51:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:53 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:51:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:53 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:51:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:53 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:51:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:53 smithi120 ceph-mon[36451]: pgmap v737: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 531 op/s 2024-06-02T15:51:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:53 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:51:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:53 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:51:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:53 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:51:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:53 smithi135 ceph-mon[36839]: pgmap v737: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 531 op/s 2024-06-02T15:51:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:53 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:51:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:53 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:51:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:53 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:51:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:53 smithi090 ceph-mon[31595]: pgmap v737: 129 pgs: 129 active+clean; 197 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 531 op/s 2024-06-02T15:51:55.479 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:55 smithi135 ceph-mon[36839]: pgmap v738: 129 pgs: 129 active+clean; 193 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-06-02T15:51:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:55 smithi120 ceph-mon[36451]: pgmap v738: 129 pgs: 129 active+clean; 193 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-06-02T15:51:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:55 smithi090 ceph-mon[31595]: pgmap v738: 129 pgs: 129 active+clean; 193 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-06-02T15:51:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:57 smithi120 ceph-mon[36451]: pgmap v739: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 652 op/s 2024-06-02T15:51:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:57 smithi135 ceph-mon[36839]: pgmap v739: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 652 op/s 2024-06-02T15:51:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:57 smithi090 ceph-mon[31595]: pgmap v739: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 652 op/s 2024-06-02T15:51:59.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:51:59 smithi120 ceph-mon[36451]: pgmap v740: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 546 op/s 2024-06-02T15:51:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:51:59 smithi135 ceph-mon[36839]: pgmap v740: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 546 op/s 2024-06-02T15:51:59.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:51:59 smithi090 ceph-mon[31595]: pgmap v740: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 546 op/s 2024-06-02T15:52:01.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:01 smithi120 ceph-mon[36451]: pgmap v741: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 554 op/s 2024-06-02T15:52:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:01 smithi135 ceph-mon[36839]: pgmap v741: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 554 op/s 2024-06-02T15:52:01.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:01 smithi090 ceph-mon[31595]: pgmap v741: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 554 op/s 2024-06-02T15:52:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:03 smithi120 ceph-mon[36451]: pgmap v742: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 621 op/s 2024-06-02T15:52:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:03 smithi135 ceph-mon[36839]: pgmap v742: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 621 op/s 2024-06-02T15:52:03.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:03 smithi090 ceph-mon[31595]: pgmap v742: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 621 op/s 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 -- 172.21.15.90:0/2815374528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb8109a50 msgr2=0x7efeb810be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 --2- 172.21.15.90:0/2815374528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb8109a50 0x7efeb810be80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efea8009900 tx=0x7efea802a3f0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 -- 172.21.15.90:0/2815374528 shutdown_connections 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 --2- 172.21.15.90:0/2815374528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8113190 0x7efeb81155e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 --2- 172.21.15.90:0/2815374528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb810c3c0 0x7efeb810c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 --2- 172.21.15.90:0/2815374528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb8109a50 0x7efeb810be80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 -- 172.21.15.90:0/2815374528 >> 172.21.15.90:0/2815374528 conn(0x7efeb806fac0 msgr2=0x7efeb8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 -- 172.21.15.90:0/2815374528 shutdown_connections 2024-06-02T15:52:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.522+0000 7efebcc00640 1 -- 172.21.15.90:0/2815374528 wait complete. 2024-06-02T15:52:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.523+0000 7efebcc00640 1 Processor -- start 2024-06-02T15:52:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.524+0000 7efebcc00640 1 -- start start 2024-06-02T15:52:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 0x7efeb81a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb810c3c0 0x7efeb81a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8119040 con 0x7efeb8109a50 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8118ec0 con 0x7efeb810c3c0 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efebcc00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81191c0 con 0x7efeb8113190 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efeb6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efeb6800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46308/0 (socket says 172.21.15.90:46308) 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efeb6800640 1 -- 172.21.15.90:0/2281301158 learned_addr learned my addr 172.21.15.90:0/2281301158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efeb5e00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 0x7efeb81a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.525+0000 7efeb5400640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb810c3c0 0x7efeb81a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 -- 172.21.15.90:0/2281301158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb810c3c0 msgr2=0x7efeb81a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb810c3c0 0x7efeb81a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 -- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 msgr2=0x7efeb81a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 0x7efeb81a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 -- 172.21.15.90:0/2281301158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81ab260 con 0x7efeb8113190 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb5e00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 0x7efeb81a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.526+0000 7efeb6800640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009c90 tx=0x7efeb000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.527+0000 7efeaea00640 1 -- 172.21.15.90:0/2281301158 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003b70 con 0x7efeb8113190 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.527+0000 7efeaea00640 1 -- 172.21.15.90:0/2281301158 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb0003d10 con 0x7efeb8113190 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.527+0000 7efeaea00640 1 -- 172.21.15.90:0/2281301158 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0010850 con 0x7efeb8113190 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.527+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb81ab550 con 0x7efeb8113190 2024-06-02T15:52:05.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.527+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb81aba50 con 0x7efeb8113190 2024-06-02T15:52:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.532+0000 7efeaea00640 1 -- 172.21.15.90:0/2281301158 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efeb0004020 con 0x7efeb8113190 2024-06-02T15:52:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.534+0000 7efeaea00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 0x7efe9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.534+0000 7efeaea00640 1 -- 172.21.15.90:0/2281301158 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efeb00e2b80 con 0x7efeb8113190 2024-06-02T15:52:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.534+0000 7efeb5e00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 0x7efe9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.535+0000 7efeb5e00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 0x7efe9c0b4f80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efea8000c00 tx=0x7efea80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.537+0000 7efebfb71740 1 Processor -- start 2024-06-02T15:52:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.537+0000 7efebfb71740 1 -- start start 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fb98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb9ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ac1fb02b90 con 0x55ac1fb93520 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ac1faf2730 con 0x55ac1fb98690 2024-06-02T15:52:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.538+0000 7efebfb71740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac1faf6770 con 0x55ac1fb92b30 2024-06-02T15:52:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46322/0 (socket says 172.21.15.90:46322) 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5e00640 1 -- 172.21.15.90:0/4027239945 learned_addr learned my addr 172.21.15.90:0/4027239945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb6800640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb9ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5e00640 1 -- 172.21.15.90:0/4027239945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 msgr2=0x55ac1fb9ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5e00640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb9ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.539+0000 7efeb5400640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fb98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efeb5e00640 1 -- 172.21.15.90:0/4027239945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 msgr2=0x55ac1fb98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efeb5e00640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fb98020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efeb5e00640 1 -- 172.21.15.90:0/4027239945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1fa8b8b0 con 0x55ac1fb92b30 2024-06-02T15:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efeb5400640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fb98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efeb5e00640 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efea8033000 tx=0x7efea803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902ab5a7074f56b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.540+0000 7efea3000640 1 -- 172.21.15.90:0/4027239945 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8035050 con 0x55ac1fb92b30 2024-06-02T15:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.541+0000 7efea3000640 1 -- 172.21.15.90:0/4027239945 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea804d070 con 0x55ac1fb92b30 2024-06-02T15:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.541+0000 7efea3000640 1 -- 172.21.15.90:0/4027239945 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8003db0 con 0x55ac1fb92b30 2024-06-02T15:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 -- 172.21.15.90:0/4027239945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 msgr2=0x55ac1fb92f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efea8033000 tx=0x7efea803c000 comp rx=0 tx=0).stop 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 -- 172.21.15.90:0/4027239945 shutdown_connections 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb9ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fb98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 --2- 172.21.15.90:0/4027239945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fb92f50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.542+0000 7efebfb71740 1 -- 172.21.15.90:0/4027239945 >> 172.21.15.90:0/4027239945 conn(0x55ac1fb8e1a0 msgr2=0x55ac1fb8e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.543+0000 7efebfb71740 1 -- 172.21.15.90:0/4027239945 shutdown_connections 2024-06-02T15:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.543+0000 7efebfb71740 1 -- 172.21.15.90:0/4027239945 wait complete. 2024-06-02T15:52:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.544+0000 7efebfb71740 1 -- start start 2024-06-02T15:52:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.544+0000 7efebfb71740 1 Processor -- start 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ac1f7d8790 con 0x55ac1fb93520 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ac1fa9abc0 con 0x55ac1fb98690 2024-06-02T15:52:05.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac1fba3df0 con 0x55ac1fb92b30 2024-06-02T15:52:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.545+0000 7efebfb71740 10 client.0 _ll_register_callbacks cb 0x55ac1fa78400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.546+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.546+0000 7efeb5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46324/0 (socket says 172.21.15.90:46324) 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.546+0000 7efeb5e00640 1 -- 172.21.15.90:0/1856184077 learned_addr learned my addr 172.21.15.90:0/1856184077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.546+0000 7efeb6800640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.546+0000 7efeb5400640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 -- 172.21.15.90:0/1856184077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 msgr2=0x55ac1fba5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 -- 172.21.15.90:0/1856184077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 msgr2=0x55ac1fba5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 -- 172.21.15.90:0/1856184077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1f807210 con 0x55ac1fb98690 2024-06-02T15:52:05.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb5400640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efeb6800640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbac680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009c60 tx=0x7efeb00a9810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec8f4a03b44ef44c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.547+0000 7efe97400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:52:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb00c9060 con 0x55ac1fb98690 2024-06-02T15:52:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efeb0004310 con 0x55ac1fb98690 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea804d070 con 0x55ac1fb98690 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb00a9d90 con 0x55ac1fb98690 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efebfb71740 10 client.26192 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ac1f8d3620 con 0x55ac1fb98690 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.548+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea804d070 con 0x55ac1fb98690 2024-06-02T15:52:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb00a5700 con 0x55ac1fb98690 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 10 client.26192 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 20 client.26192 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fbb0c20 0x55ac1fbb3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 4 client.26192 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1fb04ae0 con 0x55ac1fbb0c20 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.549+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fbb0c20 0x55ac1fbb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.550+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fbb0c20 0x55ac1fbb3080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f129beb792acd10d server_cookie=1a0fd9f7251f046e in_seq=0 out_seq=0 2024-06-02T15:52:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.550+0000 7efe97400640 10 client.26192 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.551+0000 7efe97400640 1 -- 172.21.15.90:0/1856184077 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac1fb04ae0 con 0x55ac1fbb0c20 2024-06-02T15:52:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.551+0000 7efe97400640 10 client.26192 handle_command_reply: tid=0 2024-06-02T15:52:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.551+0000 7efebfb71740 1 client.26192 shutdown 2024-06-02T15:52:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.551+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 msgr2=0x55ac1fbac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.551+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbac680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009c60 tx=0x7efeb00a9810 comp rx=0 tx=0).stop 2024-06-02T15:52:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 shutdown_connections 2024-06-02T15:52:05.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fbb0c20 0x55ac1fbb3080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbac680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 --2- 172.21.15.90:0/1856184077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efe97400640 0 client.26192 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.552+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 >> 172.21.15.90:0/1856184077 conn(0x55ac1fb8e1a0 msgr2=0x55ac1fb8d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.553+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 shutdown_connections 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.553+0000 7efebfb71740 1 -- 172.21.15.90:0/1856184077 wait complete. 2024-06-02T15:52:05.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.553+0000 7efebfb71740 20 client.26192 trim_cache size 0 max 16384 2024-06-02T15:52:05.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:05 smithi120 ceph-mon[36451]: pgmap v743: 129 pgs: 129 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 607 op/s 2024-06-02T15:52:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:05 smithi135 ceph-mon[36839]: pgmap v743: 129 pgs: 129 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 607 op/s 2024-06-02T15:52:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.586+0000 7efebfb71740 1 Processor -- start 2024-06-02T15:52:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 -- start start 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 0x55ac1fb917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb93520 0x55ac1fbac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b44f0 con 0x55ac1fb92b30 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b1d00 con 0x55ac1fb98690 2024-06-02T15:52:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efebfb71740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81afb50 con 0x55ac1fb93520 2024-06-02T15:52:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efeb6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efeb6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51664/0 (socket says 172.21.15.90:51664) 2024-06-02T15:52:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.587+0000 7efeb6800640 1 -- 172.21.15.90:0/474154807 learned_addr learned my addr 172.21.15.90:0/474154807 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb5e00640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 0x55ac1fb917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb5400640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb93520 0x55ac1fbac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 -- 172.21.15.90:0/474154807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb93520 msgr2=0x55ac1fbac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb93520 0x55ac1fbac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 -- 172.21.15.90:0/474154807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 msgr2=0x55ac1fb917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 0x55ac1fb917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 -- 172.21.15.90:0/474154807 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1fb983a0 con 0x55ac1fb98690 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb5e00640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 0x55ac1fb917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efeb6800640 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efeb00b4040 tx=0x7efeb0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206a567e49001ef8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.588+0000 7efea2600640 1 -- 172.21.15.90:0/474154807 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb00c9070 con 0x55ac1fb98690 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.589+0000 7efea2600640 1 -- 172.21.15.90:0/474154807 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb00ae140 con 0x55ac1fb98690 2024-06-02T15:52:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.589+0000 7efea2600640 1 -- 172.21.15.90:0/474154807 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0004310 con 0x55ac1fb98690 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.590+0000 7efebfb71740 1 -- 172.21.15.90:0/474154807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 msgr2=0x55ac1fb8ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.590+0000 7efebfb71740 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efeb00b4040 tx=0x7efeb0009b80 comp rx=0 tx=0).stop 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.590+0000 7efebfb71740 1 -- 172.21.15.90:0/474154807 shutdown_connections 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.590+0000 7efebfb71740 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fb8ca80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb93520 0x55ac1fbac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 --2- 172.21.15.90:0/474154807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb92b30 0x55ac1fb917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 -- 172.21.15.90:0/474154807 >> 172.21.15.90:0/474154807 conn(0x55ac1fb8e1a0 msgr2=0x55ac1fb910b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 -- 172.21.15.90:0/474154807 shutdown_connections 2024-06-02T15:52:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 -- 172.21.15.90:0/474154807 wait complete. 2024-06-02T15:52:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.591+0000 7efebfb71740 1 -- start start 2024-06-02T15:52:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 Processor -- start 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b01c0 con 0x55ac1fb93520 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b1630 con 0x55ac1fb98690 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81b0890 con 0x55ac1fb92b30 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efebfb71740 10 client.0 _ll_register_callbacks cb 0x7efeb00a5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.592+0000 7efeb6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.593+0000 7efeb6800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51666/0 (socket says 172.21.15.90:51666) 2024-06-02T15:52:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.593+0000 7efeb6800640 1 -- 172.21.15.90:0/1912741378 learned_addr learned my addr 172.21.15.90:0/1912741378 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.593+0000 7efeb6800640 1 -- 172.21.15.90:0/1912741378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 msgr2=0x55ac1fba7aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.593+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.593+0000 7efeb5400640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb6800640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb6800640 1 -- 172.21.15.90:0/1912741378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 msgr2=0x55ac1fba7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb6800640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb6800640 1 -- 172.21.15.90:0/1912741378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1fbab5a0 con 0x55ac1fb98690 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb5400640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efeb6800640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeb0003f90 tx=0x7efeb00045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ceb884b82ec0fc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efe97400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0089060 con 0x55ac1fb98690 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.594+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7efeb0004d50 con 0x55ac1fb98690 2024-06-02T15:52:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb00ae140 con 0x55ac1fb98690 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb00c9070 con 0x55ac1fb98690 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efebfb71740 10 client.26198 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ac1fbab5a0 con 0x55ac1fb98690 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb00ae140 con 0x55ac1fb98690 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.595+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb00c4040 con 0x55ac1fb98690 2024-06-02T15:52:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 10 client.26198 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 20 client.26198 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fb9f6a0 0x55ac1fba1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 4 client.26198 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.596+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1fafe2c0 con 0x55ac1fb9f6a0 2024-06-02T15:52:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.597+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fb9f6a0 0x55ac1fba1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.599+0000 7efeb5e00640 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fb9f6a0 0x55ac1fba1ac0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7de0896a4963862 server_cookie=61dbac8a32cc2a83 in_seq=0 out_seq=0 2024-06-02T15:52:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:05.599+0000 7efe97400640 10 client.26198 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:05.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:05 smithi090 ceph-mon[31595]: pgmap v743: 129 pgs: 129 active+clean; 209 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 607 op/s 2024-06-02T15:52:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.003+0000 7efe97400640 1 -- 172.21.15.90:0/1912741378 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ac1fafe2c0 con 0x55ac1fb9f6a0 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.003+0000 7efe97400640 10 client.26198 handle_command_reply: tid=0 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.003+0000 7efebfb71740 1 client.26198 shutdown 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.003+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 msgr2=0x55ac1fbaafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.003+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeb0003f90 tx=0x7efeb00045c0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 shutdown_connections 2024-06-02T15:52:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ac1fb9f6a0 0x55ac1fba1ac0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ac1fb98690 0x55ac1fbaafb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac1fb93520 0x55ac1fba7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 --2- 172.21.15.90:0/1912741378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac1fb92b30 0x55ac1fba7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 >> 172.21.15.90:0/1912741378 conn(0x55ac1fb8e1a0 msgr2=0x55ac1fb93fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efe97400640 0 client.26198 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.004+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 shutdown_connections 2024-06-02T15:52:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.005+0000 7efebfb71740 1 -- 172.21.15.90:0/1912741378 wait complete. 2024-06-02T15:52:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.005+0000 7efebfb71740 20 client.26198 trim_cache size 0 max 16384 2024-06-02T15:52:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:52:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:52:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:52:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.010+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 msgr2=0x7efe9c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.010+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 0x7efe9c0b4f80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efea8000c00 tx=0x7efea80314a0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 msgr2=0x7efeb81aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009c90 tx=0x7efeb000c6a0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 shutdown_connections 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7efe9c0b2ac0 0x7efe9c0b4f80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8113190 0x7efeb81aabf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efeb810c3c0 0x7efeb81a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 --2- 172.21.15.90:0/2281301158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8109a50 0x7efeb81a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 >> 172.21.15.90:0/2281301158 conn(0x7efeb806fac0 msgr2=0x7efeb81135d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 shutdown_connections 2024-06-02T15:52:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:06.011+0000 7efebcc00640 1 -- 172.21.15.90:0/2281301158 wait complete. 2024-06-02T15:52:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:07 smithi120 ceph-mon[36451]: pgmap v744: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 761 op/s 2024-06-02T15:52:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:07 smithi135 ceph-mon[36839]: pgmap v744: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 761 op/s 2024-06-02T15:52:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:07 smithi090 ceph-mon[31595]: pgmap v744: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 761 op/s 2024-06-02T15:52:08.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:08 smithi120 ceph-mon[36451]: pgmap v745: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 596 op/s 2024-06-02T15:52:08.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:08 smithi135 ceph-mon[36839]: pgmap v745: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 596 op/s 2024-06-02T15:52:08.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:08 smithi090 ceph-mon[31595]: pgmap v745: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 596 op/s 2024-06-02T15:52:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:11 smithi120 ceph-mon[36451]: pgmap v746: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-02T15:52:11.565 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:11 smithi135 ceph-mon[36839]: pgmap v746: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-02T15:52:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:11 smithi090 ceph-mon[31595]: pgmap v746: 129 pgs: 129 active+clean; 219 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 626 op/s 2024-06-02T15:52:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:13 smithi120 ceph-mon[36451]: pgmap v747: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 677 op/s 2024-06-02T15:52:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:13 smithi135 ceph-mon[36839]: pgmap v747: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 677 op/s 2024-06-02T15:52:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:13 smithi090 ceph-mon[31595]: pgmap v747: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 677 op/s 2024-06-02T15:52:15.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:15 smithi120 ceph-mon[36451]: pgmap v748: 129 pgs: 129 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-06-02T15:52:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:15 smithi135 ceph-mon[36839]: pgmap v748: 129 pgs: 129 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-06-02T15:52:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:15 smithi090 ceph-mon[31595]: pgmap v748: 129 pgs: 129 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 608 op/s 2024-06-02T15:52:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:17 smithi120 ceph-mon[36451]: pgmap v749: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 693 op/s 2024-06-02T15:52:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:17 smithi135 ceph-mon[36839]: pgmap v749: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 693 op/s 2024-06-02T15:52:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:17 smithi090 ceph-mon[31595]: pgmap v749: 129 pgs: 129 active+clean; 222 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 693 op/s 2024-06-02T15:52:19.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 -- 172.21.15.90:0/2373521421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0103c90 msgr2=0x7febc0106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 --2- 172.21.15.90:0/2373521421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0103c90 0x7febc0106100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7febac009900 tx=0x7febac02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 -- 172.21.15.90:0/2373521421 shutdown_connections 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 --2- 172.21.15.90:0/2373521421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0106640 0x7febc0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 --2- 172.21.15.90:0/2373521421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0103c90 0x7febc0106100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 --2- 172.21.15.90:0/2373521421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febc0101320 0x7febc0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.283+0000 7febc7800640 1 -- 172.21.15.90:0/2373521421 >> 172.21.15.90:0/2373521421 conn(0x7febc00fcf20 msgr2=0x7febc00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.284+0000 7febc7800640 1 -- 172.21.15.90:0/2373521421 shutdown_connections 2024-06-02T15:52:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.284+0000 7febc7800640 1 -- 172.21.15.90:0/2373521421 wait complete. 2024-06-02T15:52:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.285+0000 7febc7800640 1 Processor -- start 2024-06-02T15:52:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.285+0000 7febc7800640 1 -- start start 2024-06-02T15:52:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0103c90 0x7febc019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febc0106640 0x7febc01a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7febc0114c50 con 0x7febc0106640 2024-06-02T15:52:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc0114ad0 con 0x7febc0101320 2024-06-02T15:52:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc7800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febc0114dd0 con 0x7febc0103c90 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc4e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:41986/0 (socket says 172.21.15.90:41986) 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.286+0000 7febc4e00640 1 -- 172.21.15.90:0/3597906107 learned_addr learned my addr 172.21.15.90:0/3597906107 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febbfe00640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0103c90 0x7febc019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 -- 172.21.15.90:0/3597906107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0103c90 msgr2=0x7febc019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0103c90 0x7febc019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 -- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febc0106640 msgr2=0x7febc01a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febc0106640 0x7febc01a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 -- 172.21.15.90:0/3597906107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0107a70 con 0x7febc0101320 2024-06-02T15:52:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.287+0000 7febc4e00640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7febb400d950 tx=0x7febb400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.289+0000 7febbd600640 1 -- 172.21.15.90:0/3597906107 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4017070 con 0x7febc0101320 2024-06-02T15:52:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.289+0000 7febbd600640 1 -- 172.21.15.90:0/3597906107 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb4009c30 con 0x7febc0101320 2024-06-02T15:52:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.289+0000 7febbd600640 1 -- 172.21.15.90:0/3597906107 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb400f620 con 0x7febc0101320 2024-06-02T15:52:19.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.290+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc0107d60 con 0x7febc0101320 2024-06-02T15:52:19.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.290+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc0108230 con 0x7febc0101320 2024-06-02T15:52:19.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.301+0000 7febbd600640 1 -- 172.21.15.90:0/3597906107 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7febb4010460 con 0x7febc0101320 2024-06-02T15:52:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.302+0000 7febca7b1740 1 Processor -- start 2024-06-02T15:52:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.304+0000 7febbd600640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 0x7feba80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febbd600640 1 -- 172.21.15.90:0/3597906107 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7febb40e21e0 con 0x7febc0101320 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 -- start start 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febc5800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 0x7feba80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae0b30 0x5575adae0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adae8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5575ada50b90 con 0x5575adae6690 2024-06-02T15:52:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575ada40730 con 0x5575adae0b30 2024-06-02T15:52:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.305+0000 7febca7b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5575ada44770 con 0x5575adae1520 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.306+0000 7febbfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.306+0000 7febbfe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:37188/0 (socket says 172.21.15.90:37188) 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.306+0000 7febbfe00640 1 -- 172.21.15.90:0/4172439348 learned_addr learned my addr 172.21.15.90:0/4172439348 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febbfe00640 1 -- 172.21.15.90:0/4172439348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae0b30 msgr2=0x5575adae0f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febbfe00640 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae0b30 0x5575adae0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febbfe00640 1 -- 172.21.15.90:0/4172439348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 msgr2=0x5575adae8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febbfe00640 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adae8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febbfe00640 1 -- 172.21.15.90:0/4172439348 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ad9d98b0 con 0x5575adae1520 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.307+0000 7febc5800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 0x7feba80b4f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febb8005fd0 tx=0x7febb800e6c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.308+0000 7febbfe00640 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7febac009230 tx=0x7febac004160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5231955e21c5a214 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.310+0000 7febb2000640 1 -- 172.21.15.90:0/4172439348 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac035030 con 0x5575adae1520 2024-06-02T15:52:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.310+0000 7febb2000640 1 -- 172.21.15.90:0/4172439348 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac0054f0 con 0x5575adae1520 2024-06-02T15:52:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.310+0000 7febb2000640 1 -- 172.21.15.90:0/4172439348 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac003850 con 0x5575adae1520 2024-06-02T15:52:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.311+0000 7febca7b1740 1 -- 172.21.15.90:0/4172439348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 msgr2=0x5575adae6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.311+0000 7febca7b1740 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7febac009230 tx=0x7febac004160 comp rx=0 tx=0).stop 2024-06-02T15:52:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 -- 172.21.15.90:0/4172439348 shutdown_connections 2024-06-02T15:52:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adae8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae1520 0x5575adae6020 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 --2- 172.21.15.90:0/4172439348 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae0b30 0x5575adae0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 -- 172.21.15.90:0/4172439348 >> 172.21.15.90:0/4172439348 conn(0x5575adadc1a0 msgr2=0x5575adadc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 -- 172.21.15.90:0/4172439348 shutdown_connections 2024-06-02T15:52:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.312+0000 7febca7b1740 1 -- 172.21.15.90:0/4172439348 wait complete. 2024-06-02T15:52:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.313+0000 7febca7b1740 1 -- start start 2024-06-02T15:52:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.316+0000 7febca7b1740 1 Processor -- start 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae0b30 0x5575adaea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaeaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5575ad726790 con 0x5575adae0b30 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5575ad9e8bc0 con 0x5575adae1520 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5575adae95e0 con 0x5575adae6690 2024-06-02T15:52:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febca7b1740 10 client.0 _ll_register_callbacks cb 0x5575ad9c6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.319+0000 7febc5800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:37200/0 (socket says 172.21.15.90:37200) 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 -- 172.21.15.90:0/3330091156 learned_addr learned my addr 172.21.15.90:0/3330091156 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 -- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 msgr2=0x5575adaeaf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaeaf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 -- 172.21.15.90:0/3330091156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae0b30 msgr2=0x5575adaea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae0b30 0x5575adaea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.320+0000 7febc5800640 1 -- 172.21.15.90:0/3330091156 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ad755210 con 0x5575adae6690 2024-06-02T15:52:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.321+0000 7febc5800640 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7febb8012b60 tx=0x7febb8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acdf26c229593897 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.321+0000 7feba6a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:52:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8022070 con 0x5575adae6690 2024-06-02T15:52:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7febb8011af0 con 0x5575adae6690 2024-06-02T15:52:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febac0054f0 con 0x5575adae6690 2024-06-02T15:52:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7febb802e050 con 0x5575adae6690 2024-06-02T15:52:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7febca7b1740 10 client.26131 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.322+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5575ad821620 con 0x5575adae6690 2024-06-02T15:52:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac0054f0 con 0x5575adae6690 2024-06-02T15:52:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7febb800f040 con 0x5575adae6690 2024-06-02T15:52:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 10 client.26131 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 20 client.26131 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adaf6620 0x5575adaf8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 4 client.26131 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ada52ae0 con 0x5575adaf6620 2024-06-02T15:52:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.323+0000 7febc4e00640 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adaf6620 0x5575adaf8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.325+0000 7feba6a00640 10 client.26131 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.325+0000 7febc4e00640 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adaf6620 0x5575adaf8ac0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672e752619620111 server_cookie=86ed44e2ead86890 in_seq=0 out_seq=0 2024-06-02T15:52:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.327+0000 7feba6a00640 1 -- 172.21.15.90:0/3330091156 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5575ada52ae0 con 0x5575adaf6620 2024-06-02T15:52:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.327+0000 7feba6a00640 10 client.26131 handle_command_reply: tid=0 2024-06-02T15:52:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.327+0000 7febca7b1740 1 client.26131 shutdown 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 msgr2=0x5575adaf1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7febb8012b60 tx=0x7febb8004a40 comp rx=0 tx=0).stop 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 shutdown_connections 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adaf6620 0x5575adaf8ac0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae6690 0x5575adaf1ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaeaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 --2- 172.21.15.90:0/3330091156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae0b30 0x5575adaea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 >> 172.21.15.90:0/3330091156 conn(0x5575adadc1a0 msgr2=0x5575adadb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.328+0000 7feba6a00640 0 client.26131 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.329+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 shutdown_connections 2024-06-02T15:52:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.329+0000 7febca7b1740 1 -- 172.21.15.90:0/3330091156 wait complete. 2024-06-02T15:52:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.329+0000 7febca7b1740 20 client.26131 trim_cache size 0 max 16384 2024-06-02T15:52:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.363+0000 7febca7b1740 1 Processor -- start 2024-06-02T15:52:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.364+0000 7febca7b1740 1 -- start start 2024-06-02T15:52:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.364+0000 7febca7b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adadfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.364+0000 7febca7b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.364+0000 7febca7b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febca7b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7febc01bcd10 con 0x5575adae6690 2024-06-02T15:52:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febca7b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc01ba520 con 0x5575adae1520 2024-06-02T15:52:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febca7b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febc010feb0 con 0x5575adae0b30 2024-06-02T15:52:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42008/0 (socket says 172.21.15.90:42008) 2024-06-02T15:52:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.365+0000 7febbfe00640 1 -- 172.21.15.90:0/766651694 learned_addr learned my addr 172.21.15.90:0/766651694 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 -- 172.21.15.90:0/766651694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 msgr2=0x5575adadfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adadfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 -- 172.21.15.90:0/766651694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 msgr2=0x5575adaea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 -- 172.21.15.90:0/766651694 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575adae0770 con 0x5575adae1520 2024-06-02T15:52:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.366+0000 7febbfe00640 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febac02a9e0 tx=0x7febac009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22fb2768a1f916e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.368+0000 7febb1600640 1 -- 172.21.15.90:0/766651694 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac04b070 con 0x5575adae1520 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.368+0000 7febb1600640 1 -- 172.21.15.90:0/766651694 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac0046c0 con 0x5575adae1520 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.368+0000 7febb1600640 1 -- 172.21.15.90:0/766651694 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac035070 con 0x5575adae1520 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.369+0000 7febca7b1740 1 -- 172.21.15.90:0/766651694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 msgr2=0x5575adaf1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febac02a9e0 tx=0x7febac009260 comp rx=0 tx=0).stop 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 -- 172.21.15.90:0/766651694 shutdown_connections 2024-06-02T15:52:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaf1eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 --2- 172.21.15.90:0/766651694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adadfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 -- 172.21.15.90:0/766651694 >> 172.21.15.90:0/766651694 conn(0x5575adadc1a0 msgr2=0x5575adae9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 -- 172.21.15.90:0/766651694 shutdown_connections 2024-06-02T15:52:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.370+0000 7febca7b1740 1 -- 172.21.15.90:0/766651694 wait complete. 2024-06-02T15:52:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.371+0000 7febca7b1740 1 -- start start 2024-06-02T15:52:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.373+0000 7febca7b1740 1 Processor -- start 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adaef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaf07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b89e0 con 0x5575adae6690 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b9e50 con 0x5575adae1520 2024-06-02T15:52:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b90b0 con 0x5575adae0b30 2024-06-02T15:52:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.375+0000 7febca7b1740 10 client.0 _ll_register_callbacks cb 0x7febb8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.377+0000 7febc4e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adaef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.377+0000 7febbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.377+0000 7febbfe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42012/0 (socket says 172.21.15.90:42012) 2024-06-02T15:52:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.377+0000 7febbfe00640 1 -- 172.21.15.90:0/3174434843 learned_addr learned my addr 172.21.15.90:0/3174434843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febbfe00640 1 -- 172.21.15.90:0/3174434843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 msgr2=0x5575adaef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febbfe00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adaef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febbfe00640 1 -- 172.21.15.90:0/3174434843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 msgr2=0x5575adaf07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febbfe00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaf07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febbfe00640 1 -- 172.21.15.90:0/3174434843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575adaf0db0 con 0x5575adae1520 2024-06-02T15:52:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.379+0000 7febc4e00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adaef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:52:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.380+0000 7feba6a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:52:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.380+0000 7febbfe00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febac02a910 tx=0x7febac004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eaa70a934ecc59c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.381+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac035070 con 0x5575adae1520 2024-06-02T15:52:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.381+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7febac005510 con 0x5575adae1520 2024-06-02T15:52:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.382+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febac0046c0 con 0x5575adae1520 2024-06-02T15:52:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.383+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7febac04b070 con 0x5575adae1520 2024-06-02T15:52:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.383+0000 7febca7b1740 10 client.26210 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.383+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5575adaf20c0 con 0x5575adae1520 2024-06-02T15:52:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.384+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac0046c0 con 0x5575adae1520 2024-06-02T15:52:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.384+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7febac003900 con 0x5575adae1520 2024-06-02T15:52:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.384+0000 7febca7b1740 10 client.26210 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.385+0000 7febca7b1740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.385+0000 7febca7b1740 20 client.26210 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.385+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adb72830 0x5575adb74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.385+0000 7febca7b1740 4 client.26210 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:52:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.385+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575ada4c2c0 con 0x5575adb72830 2024-06-02T15:52:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.387+0000 7febc4e00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adb72830 0x5575adb74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.387+0000 7feba6a00640 10 client.26210 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:19.387+0000 7febc4e00640 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adb72830 0x5575adb74c50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9ab4cd6d3e2008 server_cookie=602356795f7894d5 in_seq=0 out_seq=0 2024-06-02T15:52:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:19 smithi120 ceph-mon[36451]: pgmap v750: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2024-06-02T15:52:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:19 smithi135 ceph-mon[36839]: pgmap v750: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2024-06-02T15:52:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:19 smithi090 ceph-mon[31595]: pgmap v750: 129 pgs: 129 active+clean; 223 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 542 op/s 2024-06-02T15:52:19.861 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:52:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:21 smithi120 ceph-mon[36451]: pgmap v751: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 514 op/s 2024-06-02T15:52:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:21 smithi135 ceph-mon[36839]: pgmap v751: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 514 op/s 2024-06-02T15:52:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:21 smithi090 ceph-mon[31595]: pgmap v751: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 514 op/s 2024-06-02T15:52:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.273+0000 7feba6a00640 1 -- 172.21.15.90:0/3174434843 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575ada4c2c0 con 0x5575adb72830 2024-06-02T15:52:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.274+0000 7feba6a00640 10 client.26210 handle_command_reply: tid=0 2024-06-02T15:52:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.274+0000 7febca7b1740 1 client.26210 shutdown 2024-06-02T15:52:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.275+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 msgr2=0x5575adaef7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.275+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febac02a910 tx=0x7febac004dd0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 shutdown_connections 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5575adb72830 0x5575adb74c50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575adae6690 0x5575adaf07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5575adae1520 0x5575adaef7b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 --2- 172.21.15.90:0/3174434843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575adae0b30 0x5575adaef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 >> 172.21.15.90:0/3174434843 conn(0x5575adadc1a0 msgr2=0x5575adadf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.276+0000 7feba6a00640 0 client.26210 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.277+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 shutdown_connections 2024-06-02T15:52:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.277+0000 7febca7b1740 1 -- 172.21.15.90:0/3174434843 wait complete. 2024-06-02T15:52:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.277+0000 7febca7b1740 20 client.26210 trim_cache size 0 max 16384 2024-06-02T15:52:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:52:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:52:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 msgr2=0x7feba80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 0x7feba80b4f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febb8005fd0 tx=0x7febb800e6c0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 msgr2=0x7febc019f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7febb400d950 tx=0x7febb400de10 comp rx=0 tx=0).stop 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 shutdown_connections 2024-06-02T15:52:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7feba80b2a70 0x7feba80b4f30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febc0106640 0x7febc01a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febc0103c90 0x7febc019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 --2- 172.21.15.90:0/3597906107 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7febc0101320 0x7febc019f200 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.280+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 >> 172.21.15.90:0/3597906107 conn(0x7febc00fcf20 msgr2=0x7febc0101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.281+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 shutdown_connections 2024-06-02T15:52:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:22.281+0000 7febc7800640 1 -- 172.21.15.90:0/3597906107 wait complete. 2024-06-02T15:52:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:23 smithi120 ceph-mon[36451]: pgmap v752: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 543 op/s 2024-06-02T15:52:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:23 smithi135 ceph-mon[36839]: pgmap v752: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 543 op/s 2024-06-02T15:52:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:23 smithi090 ceph-mon[31595]: pgmap v752: 129 pgs: 129 active+clean; 230 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 543 op/s 2024-06-02T15:52:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:25 smithi120 ceph-mon[36451]: pgmap v753: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 478 op/s 2024-06-02T15:52:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:25 smithi135 ceph-mon[36839]: pgmap v753: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 478 op/s 2024-06-02T15:52:25.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:25 smithi090 ceph-mon[31595]: pgmap v753: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 478 op/s 2024-06-02T15:52:27.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:27 smithi120 ceph-mon[36451]: pgmap v754: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 556 op/s 2024-06-02T15:52:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:27 smithi135 ceph-mon[36839]: pgmap v754: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 556 op/s 2024-06-02T15:52:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:27 smithi090 ceph-mon[31595]: pgmap v754: 129 pgs: 129 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 556 op/s 2024-06-02T15:52:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:29 smithi120 ceph-mon[36451]: pgmap v755: 129 pgs: 129 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-06-02T15:52:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:29 smithi135 ceph-mon[36839]: pgmap v755: 129 pgs: 129 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-06-02T15:52:29.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:29 smithi090 ceph-mon[31595]: pgmap v755: 129 pgs: 129 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 470 op/s 2024-06-02T15:52:30.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.817+0000 7fe4d3e00700 1 -- 172.21.15.90:0/2151557726 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc012650 con 0x7fe4d4103f90 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.818+0000 7fe4dc400700 1 -- 172.21.15.90:0/2151557726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 msgr2=0x7fe4d4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.818+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2151557726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d4110470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009f90 tx=0x7fe4cc00b330 comp rx=0 tx=0).stop 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.819+0000 7fe4dc400700 1 -- 172.21.15.90:0/2151557726 shutdown_connections 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.819+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2151557726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d4110470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.819+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2151557726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d4108500 0x7fe4d41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.819+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2151557726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d41035f0 0x7fe4d4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.819+0000 7fe4dc400700 1 -- 172.21.15.90:0/2151557726 >> 172.21.15.90:0/2151557726 conn(0x7fe4d407a6a0 msgr2=0x7fe4d407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.820+0000 7fe4dc400700 1 -- 172.21.15.90:0/2151557726 shutdown_connections 2024-06-02T15:52:30.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.821+0000 7fe4dc400700 1 -- 172.21.15.90:0/2151557726 wait complete. 2024-06-02T15:52:30.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.822+0000 7fe4dc400700 1 Processor -- start 2024-06-02T15:52:30.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.824+0000 7fe4dc400700 1 -- start start 2024-06-02T15:52:30.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d410d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d4108500 0x7fe4d410d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4115a40 con 0x7fe4d4108500 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d41158c0 con 0x7fe4d41035f0 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4dc400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4115bc0 con 0x7fe4d4103f90 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58126/0 (socket says 172.21.15.90:58126) 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4d9a00700 1 -- 172.21.15.90:0/2666769002 learned_addr learned my addr 172.21.15.90:0/2666769002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.825+0000 7fe4d9000700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d410d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4da400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d4108500 0x7fe4d410d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9a00700 1 -- 172.21.15.90:0/2666769002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 msgr2=0x7fe4d410d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d410d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9a00700 1 -- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d4108500 msgr2=0x7fe4d410d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d4108500 0x7fe4d410d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9a00700 1 -- 172.21.15.90:0/2666769002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d410e150 con 0x7fe4d41035f0 2024-06-02T15:52:30.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.826+0000 7fe4d9000700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d410d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:52:30.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.827+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800e950 tx=0x7fe4c800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.827+0000 7fe4d2000700 1 -- 172.21.15.90:0/2666769002 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800cbd0 con 0x7fe4d41035f0 2024-06-02T15:52:30.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.827+0000 7fe4d2000700 1 -- 172.21.15.90:0/2666769002 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80044e0 con 0x7fe4d41035f0 2024-06-02T15:52:30.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.827+0000 7fe4d2000700 1 -- 172.21.15.90:0/2666769002 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800cbd0 con 0x7fe4d41035f0 2024-06-02T15:52:30.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.827+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d41a1f80 con 0x7fe4d41035f0 2024-06-02T15:52:30.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.828+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d41a2470 con 0x7fe4d41035f0 2024-06-02T15:52:30.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.842+0000 7fe4e6de9b80 1 Processor -- start 2024-06-02T15:52:30.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.842+0000 7fe4d2000700 1 -- 172.21.15.90:0/2666769002 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe4c8003c80 con 0x7fe4d41035f0 2024-06-02T15:52:30.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.845+0000 7fe4e6de9b80 1 -- start start 2024-06-02T15:52:30.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.845+0000 7fe4d2000700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 0x7fe4bc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.846+0000 7fe4d2000700 1 -- 172.21.15.90:0/2666769002 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe4c80e2560 con 0x7fe4d41035f0 2024-06-02T15:52:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4da400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 0x7fe4bc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fd7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556680f2fe40 con 0x556680fd6eb0 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556680f1f9e0 con 0x556680fd1880 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.847+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556680f23a20 con 0x556680fd77d0 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4da400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fd7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4da400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:58128/0 (socket says 172.21.15.90:58128) 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4da400700 1 -- 172.21.15.90:0/1436306140 learned_addr learned my addr 172.21.15.90:0/1436306140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9000700 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1436306140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 msgr2=0x556680fd1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fd1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1436306140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 msgr2=0x556680fd1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.848+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1436306140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556680ee8420 con 0x556680fd6eb0 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.849+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fd7290 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80dfc50 tx=0x7fe4c80a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ebccdf1425206 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.850+0000 7fe4da400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 0x7fe4bc0b5090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc004d90 tx=0x7fe4cc0094c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.851+0000 7fe4c6a00700 1 -- 172.21.15.90:0/1436306140 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a9280 con 0x556680fd6eb0 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.851+0000 7fe4c6a00700 1 -- 172.21.15.90:0/1436306140 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80a9e70 con 0x556680fd6eb0 2024-06-02T15:52:30.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.851+0000 7fe4c6a00700 1 -- 172.21.15.90:0/1436306140 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a9c30 con 0x556680fd6eb0 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.852+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1436306140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 msgr2=0x556680fd7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.852+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fd7290 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80dfc50 tx=0x7fe4c80a5730 comp rx=0 tx=0).stop 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1436306140 shutdown_connections 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fd7290 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1436306140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1436306140 >> 172.21.15.90:0/1436306140 conn(0x556680effbe0 msgr2=0x556680fcc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1436306140 shutdown_connections 2024-06-02T15:52:30.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.853+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1436306140 wait complete. 2024-06-02T15:52:30.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.855+0000 7fe4e6de9b80 1 -- start start 2024-06-02T15:52:30.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.855+0000 7fe4e6de9b80 1 Processor -- start 2024-06-02T15:52:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.856+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.856+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680fdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556680e25fa0 con 0x556680fd6eb0 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556680ec0760 con 0x556680fd77d0 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556680fde240 con 0x556680fd1880 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4d9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4d9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47280/0 (socket says 172.21.15.90:47280) 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38790/0 (socket says 172.21.15.90:38790) 2024-06-02T15:52:30.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.858+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1948579193 learned_addr learned my addr 172.21.15.90:0/1948579193 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:30.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.858+0000 7fe4e6de9b80 10 client.0 _ll_register_callbacks cb 0x556680ec9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:30.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.857+0000 7fe4da400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680fdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.859+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 msgr2=0x556680fdae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.859+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680fdae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.859+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1948579193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 msgr2=0x556680fda930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.859+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.859+0000 7fe4d9a00700 1 -- 172.21.15.90:0/1948579193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556680e6f8d0 con 0x556680fd1880 2024-06-02T15:52:30.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.860+0000 7fe4d9000700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:30.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.861+0000 7fe4da400700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680fdae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:52:30.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.861+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a57f0 tx=0x7fe4c80e2440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6a3b61a91962a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.861+0000 7fe4bb400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:52:30.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.862+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80c2070 con 0x556680fd1880 2024-06-02T15:52:30.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.862+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe4c80b2350 con 0x556680fd1880 2024-06-02T15:52:30.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.862+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c80a9e70 con 0x556680fd1880 2024-06-02T15:52:30.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.862+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4c80a9280 con 0x556680fd1880 2024-06-02T15:52:30.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.863+0000 7fe4e6de9b80 10 client.26143 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:30.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.863+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556680ef0060 con 0x556680fd1880 2024-06-02T15:52:30.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.863+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80a9e70 con 0x556680fd1880 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe4c808fce0 con 0x556680fd1880 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 10 client.26143 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 20 client.26143 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556680feaca0 0x556680fed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 4 client.26143 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:30.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.865+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556680f31d90 con 0x556680feaca0 2024-06-02T15:52:30.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.866+0000 7fe4d9000700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556680feaca0 0x556680fed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.867+0000 7fe4d9000700 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556680feaca0 0x556680fed090 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72c869e1bc582ef server_cookie=66fc3de1d895f990 in_seq=0 out_seq=0 2024-06-02T15:52:30.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.867+0000 7fe4bb400700 10 client.26143 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:30.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.869+0000 7fe4bb400700 1 -- 172.21.15.90:0/1948579193 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556680f31d90 con 0x556680feaca0 2024-06-02T15:52:30.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.869+0000 7fe4bb400700 10 client.26143 handle_command_reply: tid=0 2024-06-02T15:52:30.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.869+0000 7fe4e6de9b80 1 client.26143 shutdown 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 msgr2=0x556680fdd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a57f0 tx=0x7fe4c80e2440 comp rx=0 tx=0).stop 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 shutdown_connections 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556680feaca0 0x556680fed090 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4bb400700 0 client.26143 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:30.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680fdae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd1880 0x556680fdd590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/1948579193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.870+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 >> 172.21.15.90:0/1948579193 conn(0x556680effbe0 msgr2=0x556680fd0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.871+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 shutdown_connections 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.871+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/1948579193 wait complete. 2024-06-02T15:52:30.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.871+0000 7fe4e6de9b80 20 client.26143 trim_cache size 0 max 16384 2024-06-02T15:52:30.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 Processor -- start 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 -- start start 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fcc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d41c94d0 con 0x556680fd6eb0 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4074dc0 con 0x556680fd1880 2024-06-02T15:52:30.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.923+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4072c10 con 0x556680fd77d0 2024-06-02T15:52:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.926+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.926+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60826/0 (socket says 172.21.15.90:60826) 2024-06-02T15:52:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.926+0000 7fe4d9a00700 1 -- 172.21.15.90:0/3592749805 learned_addr learned my addr 172.21.15.90:0/3592749805 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:30.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.926+0000 7fe4da400700 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.927+0000 7fe4d9a00700 1 -- 172.21.15.90:0/3592749805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 msgr2=0x556680fda710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.927+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fda710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.927+0000 7fe4d9a00700 1 -- 172.21.15.90:0/3592749805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 msgr2=0x556680fcc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.927+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fcc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.927+0000 7fe4d9a00700 1 -- 172.21.15.90:0/3592749805 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556680fd75d0 con 0x556680fd1880 2024-06-02T15:52:30.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.929+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a57a0 tx=0x7fe4c80b26b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb13e0f3bfef5ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.930+0000 7fe4c6000700 1 -- 172.21.15.90:0/3592749805 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80c2070 con 0x556680fd1880 2024-06-02T15:52:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.931+0000 7fe4c6000700 1 -- 172.21.15.90:0/3592749805 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80b2350 con 0x556680fd1880 2024-06-02T15:52:30.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.931+0000 7fe4c6000700 1 -- 172.21.15.90:0/3592749805 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8091390 con 0x556680fd1880 2024-06-02T15:52:30.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.932+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3592749805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 msgr2=0x556680fd1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.932+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80a57a0 tx=0x7fe4c80b26b0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3592749805 shutdown_connections 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd77d0 0x556680fda710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd1880 0x556680fd1260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3592749805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd6eb0 0x556680fcc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3592749805 >> 172.21.15.90:0/3592749805 conn(0x556680effbe0 msgr2=0x556680fcb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3592749805 shutdown_connections 2024-06-02T15:52:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.936+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3592749805 wait complete. 2024-06-02T15:52:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.937+0000 7fe4e6de9b80 1 -- start start 2024-06-02T15:52:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.937+0000 7fe4e6de9b80 1 Processor -- start 2024-06-02T15:52:30.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.940+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd1880 0x556680fee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd6eb0 0x556680fee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4073280 con 0x556680fd1880 2024-06-02T15:52:30.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40746f0 con 0x556680fd77d0 2024-06-02T15:52:30.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4073950 con 0x556680fd6eb0 2024-06-02T15:52:30.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4e6de9b80 10 client.0 _ll_register_callbacks cb 0x556680ec9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:30.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.941+0000 7fe4d9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd1880 0x556680fee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.942+0000 7fe4da400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.942+0000 7fe4da400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60834/0 (socket says 172.21.15.90:60834) 2024-06-02T15:52:30.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.942+0000 7fe4da400700 1 -- 172.21.15.90:0/3616213307 learned_addr learned my addr 172.21.15.90:0/3616213307 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:30.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.942+0000 7fe4d9000700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd6eb0 0x556680fee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.943+0000 7fe4da400700 1 -- 172.21.15.90:0/3616213307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd6eb0 msgr2=0x556680fee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.943+0000 7fe4da400700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd6eb0 0x556680fee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.943+0000 7fe4da400700 1 -- 172.21.15.90:0/3616213307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd1880 msgr2=0x556680fee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.943+0000 7fe4da400700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd1880 0x556680fee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.943+0000 7fe4da400700 1 -- 172.21.15.90:0/3616213307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566810547c0 con 0x556680fd77d0 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.944+0000 7fe4da400700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc007b10 tx=0x7fe4cc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a3cdf2e1c0aa37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.944+0000 7fe4bb400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.944+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc031070 con 0x556680fd77d0 2024-06-02T15:52:30.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.945+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe4cc01d680 con 0x556680fd77d0 2024-06-02T15:52:30.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.945+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c80b2350 con 0x556680fd77d0 2024-06-02T15:52:30.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.945+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc01d880 con 0x556680fd77d0 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.946+0000 7fe4e6de9b80 10 client.26228 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.946+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556681055ad0 con 0x556680fd77d0 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.947+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c80b2350 con 0x556680fd77d0 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.947+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe4cc03e020 con 0x556680fd77d0 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 20 client.26228 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5566810562c0 0x5566810586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:30.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 4 client.26228 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:52:30.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556680f2b570 con 0x5566810562c0 2024-06-02T15:52:30.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.948+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5566810562c0 0x5566810586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:30.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.950+0000 7fe4d9a00700 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5566810562c0 0x5566810586e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4f51f30fe0be63 server_cookie=91329c5cdea72a5d in_seq=0 out_seq=0 2024-06-02T15:52:30.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.950+0000 7fe4bb400700 10 client.26228 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:30.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.950+0000 7fe4bb400700 1 -- 172.21.15.90:0/3616213307 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556680f2b570 con 0x5566810562c0 2024-06-02T15:52:30.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.950+0000 7fe4bb400700 10 client.26228 handle_command_reply: tid=0 2024-06-02T15:52:30.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.951+0000 7fe4e6de9b80 1 client.26228 shutdown 2024-06-02T15:52:30.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 msgr2=0x556680feebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc007b10 tx=0x7fe4cc009920 comp rx=0 tx=0).stop 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 shutdown_connections 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5566810562c0 0x5566810586e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556680fd77d0 0x556680feebc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556680fd1880 0x556680fee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 --2- 172.21.15.90:0/3616213307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556680fd6eb0 0x556680fee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4bb400700 0 client.26228 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:30.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.952+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 >> 172.21.15.90:0/3616213307 conn(0x556680effbe0 msgr2=0x556680fda710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.953+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 shutdown_connections 2024-06-02T15:52:30.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.953+0000 7fe4e6de9b80 1 -- 172.21.15.90:0/3616213307 wait complete. 2024-06-02T15:52:30.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.953+0000 7fe4e6de9b80 20 client.26228 trim_cache size 0 max 16384 2024-06-02T15:52:30.952 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:52:30.952 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:52:30.952 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:52:30.952 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.955+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 msgr2=0x7fe4bc0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.955+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 0x7fe4bc0b5090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc004d90 tx=0x7fe4cc0094c0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.955+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 msgr2=0x7fe4d41102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.955+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800e950 tx=0x7fe4c800ed10 comp rx=0 tx=0).stop 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 shutdown_connections 2024-06-02T15:52:30.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe4bc0b2bd0 0x7fe4bc0b5090 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d4108500 0x7fe4d410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4d41035f0 0x7fe4d41102b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 --2- 172.21.15.90:0/2666769002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4d4103f90 0x7fe4d410d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 >> 172.21.15.90:0/2666769002 conn(0x7fe4d407a6a0 msgr2=0x7fe4d4101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 shutdown_connections 2024-06-02T15:52:30.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:30.956+0000 7fe4dc400700 1 -- 172.21.15.90:0/2666769002 wait complete. 2024-06-02T15:52:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:31 smithi120 ceph-mon[36451]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2024-06-02T15:52:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:31 smithi135 ceph-mon[36839]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2024-06-02T15:52:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:31 smithi090 ceph-mon[31595]: pgmap v756: 129 pgs: 129 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2024-06-02T15:52:32.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:32 smithi135 ceph-mon[36839]: pgmap v757: 129 pgs: 129 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2024-06-02T15:52:32.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:32 smithi090 ceph-mon[31595]: pgmap v757: 129 pgs: 129 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2024-06-02T15:52:32.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:32 smithi120 ceph-mon[36451]: pgmap v757: 129 pgs: 129 active+clean; 233 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2024-06-02T15:52:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:52:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:33 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:52:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:33 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:33.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:52:33.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:33.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:33 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:34 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:34 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:34 smithi090 ceph-mon[31595]: pgmap v758: 129 pgs: 129 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 419 op/s 2024-06-02T15:52:34.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:34 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:34 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:34 smithi120 ceph-mon[36451]: pgmap v758: 129 pgs: 129 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 419 op/s 2024-06-02T15:52:34.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:34 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:34 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:34.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:34 smithi135 ceph-mon[36839]: pgmap v758: 129 pgs: 129 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 419 op/s 2024-06-02T15:52:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.542+0000 7f4d35800640 1 -- 172.21.15.90:0/2826103527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d30109cc0 msgr2=0x7f4d3010c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.542+0000 7f4d35800640 1 --2- 172.21.15.90:0/2826103527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d30109cc0 0x7f4d3010c0b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009900 tx=0x7f4d1c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.543+0000 7f4d35800640 1 -- 172.21.15.90:0/2826103527 shutdown_connections 2024-06-02T15:52:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.543+0000 7f4d35800640 1 --2- 172.21.15.90:0/2826103527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d3010f160 0x7f4d30111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.543+0000 7f4d35800640 1 --2- 172.21.15.90:0/2826103527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3010c680 0x7f4d3010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.544+0000 7f4d35800640 1 --2- 172.21.15.90:0/2826103527 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d30109cc0 0x7f4d3010c0b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.544+0000 7f4d35800640 1 -- 172.21.15.90:0/2826103527 >> 172.21.15.90:0/2826103527 conn(0x7f4d3006fc80 msgr2=0x7f4d300720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.544+0000 7f4d35800640 1 -- 172.21.15.90:0/2826103527 shutdown_connections 2024-06-02T15:52:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.544+0000 7f4d35800640 1 -- 172.21.15.90:0/2826103527 wait complete. 2024-06-02T15:52:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.546+0000 7f4d35800640 1 Processor -- start 2024-06-02T15:52:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.546+0000 7f4d35800640 1 -- start start 2024-06-02T15:52:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d3010c680 0x7f4d3019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d3010f160 0x7f4d301a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30114c40 con 0x7f4d3010c680 2024-06-02T15:52:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30114ac0 con 0x7f4d3010f160 2024-06-02T15:52:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.547+0000 7f4d35800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30114dc0 con 0x7f4d30109cc0 2024-06-02T15:52:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.548+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.548+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39610/0 (socket says 172.21.15.90:39610) 2024-06-02T15:52:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.548+0000 7f4d2ea00640 1 -- 172.21.15.90:0/3835658195 learned_addr learned my addr 172.21.15.90:0/3835658195 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.549+0000 7f4d2ea00640 1 -- 172.21.15.90:0/3835658195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d3010f160 msgr2=0x7f4d301a67c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.550+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d3010f160 0x7f4d301a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.550+0000 7f4d2ea00640 1 -- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d3010c680 msgr2=0x7f4d3019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.550+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d3010c680 0x7f4d3019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.550+0000 7f4d2ea00640 1 -- 172.21.15.90:0/3835658195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d301a6e00 con 0x7f4d30109cc0 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.550+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009850 tx=0x7f4d1c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.551+0000 7f4d27400640 1 -- 172.21.15.90:0/3835658195 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c03f070 con 0x7f4d30109cc0 2024-06-02T15:52:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.551+0000 7f4d27400640 1 -- 172.21.15.90:0/3835658195 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c031c50 con 0x7f4d30109cc0 2024-06-02T15:52:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.551+0000 7f4d27400640 1 -- 172.21.15.90:0/3835658195 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c035030 con 0x7f4d30109cc0 2024-06-02T15:52:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.551+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d301a7090 con 0x7f4d30109cc0 2024-06-02T15:52:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.552+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d301a7530 con 0x7f4d30109cc0 2024-06-02T15:52:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.558+0000 7f4d27400640 1 -- 172.21.15.90:0/3835658195 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4d1c042070 con 0x7f4d30109cc0 2024-06-02T15:52:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.560+0000 7f4d27400640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 0x7f4d140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.560+0000 7f4d27400640 1 -- 172.21.15.90:0/3835658195 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d1c103e80 con 0x7f4d30109cc0 2024-06-02T15:52:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.562+0000 7f4d2e000640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 0x7f4d140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.562+0000 7f4d3864f740 1 Processor -- start 2024-06-02T15:52:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.562+0000 7f4d3864f740 1 -- start start 2024-06-02T15:52:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfed7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfeda330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653dfe39b90 con 0x5653dfed7ea0 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653dfe29730 con 0x5653dfed2340 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.563+0000 7f4d3864f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653dfe2d770 con 0x5653dfed2d30 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60868/0 (socket says 172.21.15.90:60868) 2024-06-02T15:52:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 -- 172.21.15.90:0/4020798472 learned_addr learned my addr 172.21.15.90:0/4020798472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 -- 172.21.15.90:0/4020798472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 msgr2=0x5653dfed7830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfed7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 -- 172.21.15.90:0/4020798472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 msgr2=0x5653dfeda330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfeda330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.564+0000 7f4d2ea00640 1 -- 172.21.15.90:0/4020798472 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dfdc28b0 con 0x5653dfed2340 2024-06-02T15:52:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.565+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cb020 tx=0x7f4d1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2051e7df418c2c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.565+0000 7f4d1be00640 1 -- 172.21.15.90:0/4020798472 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c049040 con 0x5653dfed2340 2024-06-02T15:52:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.565+0000 7f4d1be00640 1 -- 172.21.15.90:0/4020798472 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c0cb790 con 0x5653dfed2340 2024-06-02T15:52:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.565+0000 7f4d1be00640 1 -- 172.21.15.90:0/4020798472 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c049040 con 0x5653dfed2340 2024-06-02T15:52:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.567+0000 7f4d2e000640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 0x7f4d140b4eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d301a0720 tx=0x7f4d20005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.567+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020798472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 msgr2=0x5653dfed2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.567+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cb020 tx=0x7f4d1c009260 comp rx=0 tx=0).stop 2024-06-02T15:52:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.569+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020798472 shutdown_connections 2024-06-02T15:52:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.569+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfeda330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.569+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfed7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.569+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020798472 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfed2760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.569+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020798472 >> 172.21.15.90:0/4020798472 conn(0x5653dfec51a0 msgr2=0x5653dfec55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.571+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020798472 shutdown_connections 2024-06-02T15:52:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.571+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020798472 wait complete. 2024-06-02T15:52:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.572+0000 7f4d3864f740 1 -- start start 2024-06-02T15:52:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.572+0000 7f4d3864f740 1 Processor -- start 2024-06-02T15:52:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.573+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dfecac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.573+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.573+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfed01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.574+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.574+0000 7f4d2e000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60878/0 (socket says 172.21.15.90:60878) 2024-06-02T15:52:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.574+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dfecac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.574+0000 7f4d2ea00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dfecac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39614/0 (socket says 172.21.15.90:39614) 2024-06-02T15:52:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d2e000640 1 -- 172.21.15.90:0/4020120977 learned_addr learned my addr 172.21.15.90:0/4020120977 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d2f400640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfed01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653dfb0f790 con 0x5653dfed7ea0 2024-06-02T15:52:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653dfdd1bc0 con 0x5653dfed2d30 2024-06-02T15:52:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653dfec99f0 con 0x5653dfed2340 2024-06-02T15:52:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.575+0000 7f4d3864f740 10 client.0 _ll_register_callbacks cb 0x5653dfdaf400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 -- 172.21.15.90:0/4020120977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 msgr2=0x5653dfecac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dfecac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 -- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 msgr2=0x5653dfecb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dfb3e210 con 0x5653dfed7ea0 2024-06-02T15:52:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2e000640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.576+0000 7f4d2f400640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfed01f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800afd0 tx=0x7f4d2800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748066a6e1a6e105 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.577+0000 7f4d18c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:52:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.579+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018020 con 0x5653dfed7ea0 2024-06-02T15:52:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.580+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d28004d20 con 0x5653dfed7ea0 2024-06-02T15:52:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.581+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c0cb790 con 0x5653dfed7ea0 2024-06-02T15:52:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.581+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4d2800aa90 con 0x5653dfed7ea0 2024-06-02T15:52:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.581+0000 7f4d3864f740 10 client.15960 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.581+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5653dfc0a620 con 0x5653dfed7ea0 2024-06-02T15:52:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.582+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d1c0cb790 con 0x5653dfed7ea0 2024-06-02T15:52:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4d28013070 con 0x5653dfed7ea0 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 10 client.15960 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 20 client.15960 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dfef0280 0x5653dfef2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 4 client.15960 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.583+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653dfe3bae0 con 0x5653dfef0280 2024-06-02T15:52:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.584+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dfef0280 0x5653dfef2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.585+0000 7f4d18c00640 10 client.15960 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.586+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dfef0280 0x5653dfef2700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43299e6bb651bed server_cookie=87502d1a2443addb in_seq=0 out_seq=0 2024-06-02T15:52:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.586+0000 7f4d18c00640 1 -- 172.21.15.90:0/4020120977 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653dfe3bae0 con 0x5653dfef0280 2024-06-02T15:52:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.587+0000 7f4d18c00640 10 client.15960 handle_command_reply: tid=0 2024-06-02T15:52:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.587+0000 7f4d3864f740 1 client.15960 shutdown 2024-06-02T15:52:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.587+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 msgr2=0x5653dfed01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.587+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfed01f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800afd0 tx=0x7f4d2800a750 comp rx=0 tx=0).stop 2024-06-02T15:52:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 shutdown_connections 2024-06-02T15:52:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dfef0280 0x5653dfef2700 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfed01f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dfecb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 --2- 172.21.15.90:0/4020120977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dfecac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d18c00640 0 client.15960 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.588+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 >> 172.21.15.90:0/4020120977 conn(0x5653dfec51a0 msgr2=0x5653dfec4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.589+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 shutdown_connections 2024-06-02T15:52:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.591+0000 7f4d3864f740 1 -- 172.21.15.90:0/4020120977 wait complete. 2024-06-02T15:52:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.591+0000 7f4d3864f740 20 client.15960 trim_cache size 0 max 16384 2024-06-02T15:52:35.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:52:35.598 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:52:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.637+0000 7f4d3864f740 1 Processor -- start 2024-06-02T15:52:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 -- start start 2024-06-02T15:52:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfec3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfec3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfec4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301b4570 con 0x5653dfed7ea0 2024-06-02T15:52:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301b1d80 con 0x5653dfed2340 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.638+0000 7f4d3864f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301afbd0 con 0x5653dfed2d30 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.639+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfec4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.639+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfec4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55816/0 (socket says 172.21.15.90:55816) 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.639+0000 7f4d2f400640 1 -- 172.21.15.90:0/143184498 learned_addr learned my addr 172.21.15.90:0/143184498 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.639+0000 7f4d2e000640 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfec3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.639+0000 7f4d2e000640 1 -- 172.21.15.90:0/143184498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 msgr2=0x5653dfec3a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.640+0000 7f4d2e000640 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfec3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.640+0000 7f4d2e000640 1 -- 172.21.15.90:0/143184498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 msgr2=0x5653dfec4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.640+0000 7f4d2e000640 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfec4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.640+0000 7f4d2e000640 1 -- 172.21.15.90:0/143184498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dfec9b30 con 0x5653dfed2d30 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.640+0000 7f4d2e000640 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfec3fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d20005eb0 tx=0x7f4d2000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78768a46fb081155 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.641+0000 7f4d1b400640 1 -- 172.21.15.90:0/143184498 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20021070 con 0x5653dfed2d30 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.641+0000 7f4d1b400640 1 -- 172.21.15.90:0/143184498 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d20015040 con 0x5653dfed2d30 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.641+0000 7f4d1b400640 1 -- 172.21.15.90:0/143184498 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000e830 con 0x5653dfed2d30 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.642+0000 7f4d3864f740 1 -- 172.21.15.90:0/143184498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 msgr2=0x5653dfec3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.642+0000 7f4d3864f740 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfec3fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d20005eb0 tx=0x7f4d2000f000 comp rx=0 tx=0).stop 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 -- 172.21.15.90:0/143184498 shutdown_connections 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dfec4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2d30 0x5653dfec3fc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 --2- 172.21.15.90:0/143184498 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2340 0x5653dfec3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 -- 172.21.15.90:0/143184498 >> 172.21.15.90:0/143184498 conn(0x5653dfec51a0 msgr2=0x5653dfecac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 -- 172.21.15.90:0/143184498 shutdown_connections 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.643+0000 7f4d3864f740 1 -- 172.21.15.90:0/143184498 wait complete. 2024-06-02T15:52:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.644+0000 7f4d3864f740 1 -- start start 2024-06-02T15:52:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.644+0000 7f4d3864f740 1 Processor -- start 2024-06-02T15:52:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.644+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dff013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dff01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301b0240 con 0x5653dfed7ea0 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301b16b0 con 0x5653dfed2d30 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d301b0910 con 0x5653dfed2340 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.645+0000 7f4d3864f740 10 client.0 _ll_register_callbacks cb 0x7f4d28009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.646+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.646+0000 7f4d2f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55828/0 (socket says 172.21.15.90:55828) 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.646+0000 7f4d2f400640 1 -- 172.21.15.90:0/404952868 learned_addr learned my addr 172.21.15.90:0/404952868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.646+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dff013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2e000640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dff01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2f400640 1 -- 172.21.15.90:0/404952868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 msgr2=0x5653dff013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2f400640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dff013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2f400640 1 -- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 msgr2=0x5653dff01910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2f400640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dff01910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.647+0000 7f4d2f400640 1 -- 172.21.15.90:0/404952868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dff591d0 con 0x5653dfed7ea0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.648+0000 7f4d2f400640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000f80 tx=0x7f4d28005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f050e4f8dced74 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.648+0000 7f4d18c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.649+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800c760 con 0x5653dfed7ea0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.649+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f4d2800c900 con 0x5653dfed7ea0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.649+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d20015040 con 0x5653dfed7ea0 2024-06-02T15:52:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.650+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4d280093e0 con 0x5653dfed7ea0 2024-06-02T15:52:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.650+0000 7f4d3864f740 10 client.15966 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.650+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5653dff5a580 con 0x5653dfed7ea0 2024-06-02T15:52:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.651+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d20015040 con 0x5653dfed7ea0 2024-06-02T15:52:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4d28013040 con 0x5653dfed7ea0 2024-06-02T15:52:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 10 client.15966 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:35.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 20 client.15966 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dff5afc0 0x5653dff5d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 4 client.15966 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:52:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.652+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653dfe352c0 con 0x5653dff5afc0 2024-06-02T15:52:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.653+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dff5afc0 0x5653dff5d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.653+0000 7f4d2ea00640 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dff5afc0 0x5653dff5d3e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6597b61a000d6d9 server_cookie=c79add2b71f48e4f in_seq=0 out_seq=0 2024-06-02T15:52:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.653+0000 7f4d18c00640 10 client.15966 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.817+0000 7f4d18c00640 1 -- 172.21.15.90:0/404952868 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653dfe352c0 con 0x5653dff5afc0 2024-06-02T15:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.817+0000 7f4d18c00640 10 client.15966 handle_command_reply: tid=0 2024-06-02T15:52:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.818+0000 7f4d3864f740 1 client.15966 shutdown 2024-06-02T15:52:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.818+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 msgr2=0x5653dff58c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.818+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000f80 tx=0x7f4d28005c50 comp rx=0 tx=0).stop 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 shutdown_connections 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653dff5afc0 0x5653dff5d3e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653dfed7ea0 0x5653dff58c90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653dfed2d30 0x5653dff01910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 --2- 172.21.15.90:0/404952868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dfed2340 0x5653dff013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 >> 172.21.15.90:0/404952868 conn(0x5653dfec51a0 msgr2=0x5653dfed3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.819+0000 7f4d18c00640 0 client.15966 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.820+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 shutdown_connections 2024-06-02T15:52:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.820+0000 7f4d3864f740 1 -- 172.21.15.90:0/404952868 wait complete. 2024-06-02T15:52:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.820+0000 7f4d3864f740 20 client.15966 trim_cache size 0 max 16384 2024-06-02T15:52:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:52:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:52:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:52:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.824+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 msgr2=0x7f4d140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.824+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 0x7f4d140b4eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d301a0720 tx=0x7f4d20005c70 comp rx=0 tx=0).stop 2024-06-02T15:52:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.824+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 msgr2=0x7f4d3019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.824+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009850 tx=0x7f4d1c003b90 comp rx=0 tx=0).stop 2024-06-02T15:52:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 shutdown_connections 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4d140b29f0 0x7f4d140b4eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d3010f160 0x7f4d301a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d3010c680 0x7f4d3019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 --2- 172.21.15.90:0/3835658195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d30109cc0 0x7f4d3019f210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.825+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 >> 172.21.15.90:0/3835658195 conn(0x7f4d3006fc80 msgr2=0x7f4d3010a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.826+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 shutdown_connections 2024-06-02T15:52:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:35.826+0000 7f4d35800640 1 -- 172.21.15.90:0/3835658195 wait complete. 2024-06-02T15:52:35.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:52:35.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:52:35.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:35 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:36.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:52:36.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:52:36.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:35 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:36.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:52:36.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:52:36.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:35 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:52:36.308 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:52:37.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:36 smithi120 ceph-mon[36451]: pgmap v759: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 481 op/s 2024-06-02T15:52:37.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:36 smithi135 ceph-mon[36839]: pgmap v759: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 481 op/s 2024-06-02T15:52:37.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:36 smithi090 ceph-mon[31595]: pgmap v759: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 481 op/s 2024-06-02T15:52:39.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:39 smithi090 ceph-mon[31595]: pgmap v760: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.435+0000 7f1f1f800700 1 -- 172.21.15.90:0/3650873208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102440 msgr2=0x7f1f181028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.435+0000 7f1f1f800700 1 --2- 172.21.15.90:0/3650873208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102440 0x7f1f181028c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f0001a9a0 tx=0x7f1f0001acb0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.435+0000 7f1f1f800700 1 -- 172.21.15.90:0/3650873208 shutdown_connections 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 --2- 172.21.15.90:0/3650873208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18102e00 0x7f1f1810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 --2- 172.21.15.90:0/3650873208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f18108520 0x7f1f18108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 --2- 172.21.15.90:0/3650873208 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102440 0x7f1f181028c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 -- 172.21.15.90:0/3650873208 >> 172.21.15.90:0/3650873208 conn(0x7f1f180793f0 msgr2=0x7f1f18079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 -- 172.21.15.90:0/3650873208 shutdown_connections 2024-06-02T15:52:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.436+0000 7f1f1f800700 1 -- 172.21.15.90:0/3650873208 wait complete. 2024-06-02T15:52:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.437+0000 7f1f1f800700 1 Processor -- start 2024-06-02T15:52:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.437+0000 7f1f1f800700 1 -- start start 2024-06-02T15:52:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f18102440 0x7f1f1810e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:39.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:39.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18108520 0x7f1f18197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:39.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18070840 con 0x7f1f18102440 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180706c0 con 0x7f1f18102e00 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f1f800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180709c0 con 0x7f1f18108520 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.438+0000 7f1f0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:60910/0 (socket says 172.21.15.90:60910) 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 -- 172.21.15.90:0/622715994 learned_addr learned my addr 172.21.15.90:0/622715994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 -- 172.21.15.90:0/622715994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18108520 msgr2=0x7f1f18197fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18108520 0x7f1f18197fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 -- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f18102440 msgr2=0x7f1f1810e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f18102440 0x7f1f1810e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 -- 172.21.15.90:0/622715994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181986f0 con 0x7f1f18102e00 2024-06-02T15:52:39.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.439+0000 7f1f0fe00700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f00028990 tx=0x7f1f00028a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:39.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.440+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00036070 con 0x7f1f18102e00 2024-06-02T15:52:39.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.440+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f00003680 con 0x7f1f18102e00 2024-06-02T15:52:39.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.440+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00031da0 con 0x7f1f18102e00 2024-06-02T15:52:39.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.440+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f18198980 con 0x7f1f18102e00 2024-06-02T15:52:39.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.440+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181a02a0 con 0x7f1f18102e00 2024-06-02T15:52:39.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.442+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1806a230 con 0x7f1f18102e00 2024-06-02T15:52:39.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.446+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1f00039070 con 0x7f1f18102e00 2024-06-02T15:52:39.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.449+0000 7f1f16000700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 0x7f1efc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:39.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.449+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f1f000f3cd0 con 0x7f1f18102e00 2024-06-02T15:52:39.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.450+0000 7f1f1ce00700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 0x7f1efc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:39.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.451+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f00024070 con 0x7f1f18102e00 2024-06-02T15:52:39.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.453+0000 7f1f1ce00700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 0x7f1efc0b4ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800ab90 tx=0x7f1f08009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:39.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:39 smithi120 ceph-mon[36451]: pgmap v760: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2024-06-02T15:52:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:39 smithi135 ceph-mon[36839]: pgmap v760: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2024-06-02T15:52:39.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.970+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f181a0470 con 0x7f1f18102e00 2024-06-02T15:52:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.971+0000 7f1f16000700 1 -- 172.21.15.90:0/622715994 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11887 (secure 0 0 0) 0x7f1f000bb750 con 0x7f1f18102e00 2024-06-02T15:52:39.969 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:52:39.969 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:51:09.482439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 msgr2=0x7f1efc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 0x7f1efc0b4ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800ab90 tx=0x7f1f08009250 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 msgr2=0x7f1f1810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f00028990 tx=0x7f1f00028a70 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 shutdown_connections 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1efc0b2a10 0x7f1efc0b4ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f18108520 0x7f1f18197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f18102440 0x7f1f1810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 --2- 172.21.15.90:0/622715994 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f18102e00 0x7f1f1810e7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:39.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.975+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 >> 172.21.15.90:0/622715994 conn(0x7f1f180793f0 msgr2=0x7f1f18100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:39.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.976+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 shutdown_connections 2024-06-02T15:52:39.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:39.976+0000 7f1f1f800700 1 -- 172.21.15.90:0/622715994 wait complete. 2024-06-02T15:52:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-06-02T15:52:40.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:40 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/622715994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:52:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:40 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/622715994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:52:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:40 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/622715994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:52:41.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:41 smithi120 ceph-mon[36451]: pgmap v761: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-06-02T15:52:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:41 smithi135 ceph-mon[36839]: pgmap v761: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-06-02T15:52:41.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:41 smithi090 ceph-mon[31595]: pgmap v761: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-06-02T15:52:43.056 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:42 smithi120 ceph-mon[36451]: pgmap v762: 129 pgs: 129 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-06-02T15:52:43.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:42 smithi135 ceph-mon[36839]: pgmap v762: 129 pgs: 129 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-06-02T15:52:43.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:42 smithi090 ceph-mon[31595]: pgmap v762: 129 pgs: 129 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 332 op/s 2024-06-02T15:52:44.157 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:52:44.814 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:52:45.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:45 smithi090 ceph-mon[31595]: pgmap v763: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-06-02T15:52:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:45 smithi120 ceph-mon[36451]: pgmap v763: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-06-02T15:52:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:45 smithi135 ceph-mon[36839]: pgmap v763: 129 pgs: 129 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-06-02T15:52:47.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:47 smithi090 ceph-mon[31595]: pgmap v764: 129 pgs: 129 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-06-02T15:52:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:47 smithi120 ceph-mon[36451]: pgmap v764: 129 pgs: 129 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-06-02T15:52:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:47 smithi135 ceph-mon[36839]: pgmap v764: 129 pgs: 129 active+clean; 247 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 292 op/s 2024-06-02T15:52:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 -- 172.21.15.90:0/1999010746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2720076a90 msgr2=0x7f2720074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 --2- 172.21.15.90:0/1999010746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2720076a90 0x7f2720074ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2710009930 tx=0x7f271002a440 comp rx=0 tx=0).stop 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 -- 172.21.15.90:0/1999010746 shutdown_connections 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 --2- 172.21.15.90:0/1999010746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f272010eeb0 0x7f27201112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 --2- 172.21.15.90:0/1999010746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f27200754b0 0x7f2720075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 --2- 172.21.15.90:0/1999010746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2720076a90 0x7f2720074ee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 -- 172.21.15.90:0/1999010746 >> 172.21.15.90:0/1999010746 conn(0x7f27200fde50 msgr2=0x7f2720100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.021+0000 7f2725600640 1 -- 172.21.15.90:0/1999010746 shutdown_connections 2024-06-02T15:52:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.022+0000 7f2725600640 1 -- 172.21.15.90:0/1999010746 wait complete. 2024-06-02T15:52:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.023+0000 7f2725600640 1 Processor -- start 2024-06-02T15:52:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.023+0000 7f2725600640 1 -- start start 2024-06-02T15:52:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2720076a90 0x7f272019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f272010eeb0 0x7f27201a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2720114a00 con 0x7f27200754b0 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2720114880 con 0x7f272010eeb0 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2725600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2720114b80 con 0x7f2720076a90 2024-06-02T15:52:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46366/0 (socket says 172.21.15.90:46366) 2024-06-02T15:52:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f271fe00640 1 -- 172.21.15.90:0/2677075685 learned_addr learned my addr 172.21.15.90:0/2677075685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f271f400640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2720076a90 0x7f272019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.024+0000 7f2724c00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f272010eeb0 0x7f27201a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 -- 172.21.15.90:0/2677075685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2720076a90 msgr2=0x7f272019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2720076a90 0x7f272019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 -- 172.21.15.90:0/2677075685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f272010eeb0 msgr2=0x7f27201a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f272010eeb0 0x7f27201a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 -- 172.21.15.90:0/2677075685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27201a6d80 con 0x7f27200754b0 2024-06-02T15:52:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.025+0000 7f271fe00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2710009a50 tx=0x7f27100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.026+0000 7f271cc00640 1 -- 172.21.15.90:0/2677075685 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271003f030 con 0x7f27200754b0 2024-06-02T15:52:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.026+0000 7f271cc00640 1 -- 172.21.15.90:0/2677075685 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2710003cb0 con 0x7f27200754b0 2024-06-02T15:52:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.026+0000 7f271cc00640 1 -- 172.21.15.90:0/2677075685 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710035070 con 0x7f27200754b0 2024-06-02T15:52:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.026+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201a7010 con 0x7f27200754b0 2024-06-02T15:52:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.026+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a74b0 con 0x7f27200754b0 2024-06-02T15:52:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.031+0000 7f2729b90740 1 Processor -- start 2024-06-02T15:52:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f271cc00640 1 -- 172.21.15.90:0/2677075685 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2710042030 con 0x7f27200754b0 2024-06-02T15:52:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 -- start start 2024-06-02T15:52:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142083cf0 0x559142084110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142089850 0x55914208bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559141ff3b90 con 0x559142089850 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559141fe3730 con 0x559142083cf0 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f2729b90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559141fe7770 con 0x5591420846e0 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.032+0000 7f271fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55346/0 (socket says 172.21.15.90:55346) 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 -- 172.21.15.90:0/418422392 learned_addr learned my addr 172.21.15.90:0/418422392 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271f400640 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142089850 0x55914208bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 -- 172.21.15.90:0/418422392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142083cf0 msgr2=0x559142084110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142083cf0 0x559142084110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 -- 172.21.15.90:0/418422392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142089850 msgr2=0x55914208bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142089850 0x55914208bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.033+0000 7f271fe00640 1 -- 172.21.15.90:0/418422392 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559141f7c8b0 con 0x5591420846e0 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.034+0000 7f271fe00640 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2710106e40 tx=0x7f2710106f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d2d442b78f94cd6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.034+0000 7f2715200640 1 -- 172.21.15.90:0/418422392 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271010bb30 con 0x5591420846e0 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.034+0000 7f2715200640 1 -- 172.21.15.90:0/418422392 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2710110070 con 0x5591420846e0 2024-06-02T15:52:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.034+0000 7f2715200640 1 -- 172.21.15.90:0/418422392 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710109670 con 0x5591420846e0 2024-06-02T15:52:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.035+0000 7f271cc00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 0x7f27040b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.035+0000 7f271cc00640 1 -- 172.21.15.90:0/2677075685 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2710102d90 con 0x7f27200754b0 2024-06-02T15:52:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.036+0000 7f2724c00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 0x7f27040b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.037+0000 7f2729b90740 1 -- 172.21.15.90:0/418422392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 msgr2=0x5591420891e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.037+0000 7f2724c00640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 0x7f27040b4e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x559142084f40 tx=0x7f2718009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.037+0000 7f2729b90740 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2710106e40 tx=0x7f2710106f30 comp rx=0 tx=0).stop 2024-06-02T15:52:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 -- 172.21.15.90:0/418422392 shutdown_connections 2024-06-02T15:52:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142089850 0x55914208bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x5591420891e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 --2- 172.21.15.90:0/418422392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142083cf0 0x559142084110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 -- 172.21.15.90:0/418422392 >> 172.21.15.90:0/418422392 conn(0x55914207f360 msgr2=0x55914207f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 -- 172.21.15.90:0/418422392 shutdown_connections 2024-06-02T15:52:49.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.038+0000 7f2729b90740 1 -- 172.21.15.90:0/418422392 wait complete. 2024-06-02T15:52:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.039+0000 7f2729b90740 1 -- start start 2024-06-02T15:52:49.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.039+0000 7f2729b90740 1 Processor -- start 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559142083cf0 0x55914208d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591420846e0 0x55914208ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559141cc9790 con 0x5591420846e0 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559141f8bbc0 con 0x559142089850 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55914208c5c0 con 0x559142083cf0 2024-06-02T15:52:49.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.040+0000 7f2729b90740 10 client.0 _ll_register_callbacks cb 0x559141f69400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:44422/0 (socket says 172.21.15.90:44422) 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 -- 172.21.15.90:0/1642230731 learned_addr learned my addr 172.21.15.90:0/1642230731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 -- 172.21.15.90:0/1642230731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559142083cf0 msgr2=0x55914208d880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559142083cf0 0x55914208d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 -- 172.21.15.90:0/1642230731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591420846e0 msgr2=0x55914208ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591420846e0 0x55914208ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 -- 172.21.15.90:0/1642230731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559141cf8210 con 0x559142089850 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.041+0000 7f2724c00640 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2718003fd0 tx=0x7f2718004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea2ce56aaa68b3a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.042+0000 7f2709600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.042+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271800fd50 con 0x559142089850 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.042+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f271801e420 con 0x559142089850 2024-06-02T15:52:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.042+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2710110070 con 0x559142089850 2024-06-02T15:52:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.042+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2718004810 con 0x559142089850 2024-06-02T15:52:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 10 client.26255 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559141dc4620 con 0x559142089850 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2710110070 con 0x559142089850 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f27180306f0 con 0x559142089850 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 10 client.26255 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 20 client.26255 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142099250 0x55914209b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 4 client.26255 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.043+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559141ff5ae0 con 0x559142099250 2024-06-02T15:52:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.044+0000 7f271fe00640 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142099250 0x55914209b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.044+0000 7f271fe00640 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142099250 0x55914209b6f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c048ea26e4ec0dd server_cookie=236026f3edba8478 in_seq=0 out_seq=0 2024-06-02T15:52:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.045+0000 7f2709600640 10 client.26255 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.054+0000 7f2709600640 1 -- 172.21.15.90:0/1642230731 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559141ff5ae0 con 0x559142099250 2024-06-02T15:52:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.054+0000 7f2709600640 10 client.26255 handle_command_reply: tid=0 2024-06-02T15:52:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.054+0000 7f2729b90740 1 client.26255 shutdown 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 msgr2=0x559142094e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2718003fd0 tx=0x7f2718004ab0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 shutdown_connections 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142099250 0x55914209b6f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142094e00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591420846e0 0x55914208ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 --2- 172.21.15.90:0/1642230731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559142083cf0 0x55914208d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 >> 172.21.15.90:0/1642230731 conn(0x55914207f360 msgr2=0x55914207e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.055+0000 7f2709600640 0 client.26255 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.056+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 shutdown_connections 2024-06-02T15:52:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.056+0000 7f2729b90740 1 -- 172.21.15.90:0/1642230731 wait complete. 2024-06-02T15:52:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.056+0000 7f2729b90740 20 client.26255 trim_cache size 0 max 16384 2024-06-02T15:52:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 Processor -- start 2024-06-02T15:52:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 -- start start 2024-06-02T15:52:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142094dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142085180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2700012640 con 0x559142083cf0 2024-06-02T15:52:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f270000fe50 con 0x559142089850 2024-06-02T15:52:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2729b90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f270000dca0 con 0x5591420846e0 2024-06-02T15:52:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f2724c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142085180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46390/0 (socket says 172.21.15.90:46390) 2024-06-02T15:52:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f271fe00640 1 -- 172.21.15.90:0/2057373571 learned_addr learned my addr 172.21.15.90:0/2057373571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.082+0000 7f271f400640 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142094dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 -- 172.21.15.90:0/2057373571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 msgr2=0x559142094dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142094dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 -- 172.21.15.90:0/2057373571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 msgr2=0x559142082980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 -- 172.21.15.90:0/2057373571 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914208c960 con 0x559142089850 2024-06-02T15:52:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f271fe00640 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:52:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f2724c00640 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142085180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2718030ac0 tx=0x7f2718026480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b6b39664b86287 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.083+0000 7f270be00640 1 -- 172.21.15.90:0/2057373571 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718036040 con 0x559142089850 2024-06-02T15:52:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f270be00640 1 -- 172.21.15.90:0/2057373571 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2718003e10 con 0x559142089850 2024-06-02T15:52:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f270be00640 1 -- 172.21.15.90:0/2057373571 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718026750 con 0x559142089850 2024-06-02T15:52:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 -- 172.21.15.90:0/2057373571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 msgr2=0x559142085180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142085180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2718030ac0 tx=0x7f2718026480 comp rx=0 tx=0).stop 2024-06-02T15:52:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 -- 172.21.15.90:0/2057373571 shutdown_connections 2024-06-02T15:52:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142085180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142094dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 --2- 172.21.15.90:0/2057373571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142082980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 -- 172.21.15.90:0/2057373571 >> 172.21.15.90:0/2057373571 conn(0x55914207f360 msgr2=0x559142082270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.084+0000 7f2729b90740 1 -- 172.21.15.90:0/2057373571 shutdown_connections 2024-06-02T15:52:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 -- 172.21.15.90:0/2057373571 wait complete. 2024-06-02T15:52:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 -- start start 2024-06-02T15:52:49.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 Processor -- start 2024-06-02T15:52:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142092380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.085+0000 7f2729b90740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142093320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f2729b90740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f270000e310 con 0x559142083cf0 2024-06-02T15:52:49.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f2729b90740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f270000f780 con 0x559142089850 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f2729b90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f270000e9e0 con 0x5591420846e0 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f2729b90740 10 client.0 _ll_register_callbacks cb 0x7f2718009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271f400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142092380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46396/0 (socket says 172.21.15.90:46396) 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 -- 172.21.15.90:0/1462318077 learned_addr learned my addr 172.21.15.90:0/1462318077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f2724c00640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142093320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 -- 172.21.15.90:0/1462318077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 msgr2=0x559142092380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142092380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 -- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 msgr2=0x559142093320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142093320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.086+0000 7f271fe00640 1 -- 172.21.15.90:0/1462318077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914207f190 con 0x559142083cf0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f2724c00640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142093320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f271fe00640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2710006010 tx=0x7f2710102fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5228c78e9e0fae8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f2709600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27100e4060 con 0x559142083cf0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f2710100590 con 0x559142083cf0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.087+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559141fef2c0 con 0x559142083cf0 2024-06-02T15:52:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.088+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f27100e9050 con 0x559142083cf0 2024-06-02T15:52:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.088+0000 7f2729b90740 10 client.15981 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.088+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559142094a50 con 0x559142083cf0 2024-06-02T15:52:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.089+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559141fef2c0 con 0x559142083cf0 2024-06-02T15:52:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.089+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f2710108050 con 0x559142083cf0 2024-06-02T15:52:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 10 client.15981 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 20 client.15981 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142103ed0 0x5591421062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 4 client.15981 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.090+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559141f694b0 con 0x559142103ed0 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.091+0000 7f271f400640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142103ed0 0x5591421062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.092+0000 7f271f400640 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142103ed0 0x5591421062c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52fa779227d1133 server_cookie=6c51e1de36419b86 in_seq=0 out_seq=0 2024-06-02T15:52:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.092+0000 7f2709600640 10 client.15981 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2709600640 1 -- 172.21.15.90:0/1462318077 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559141f694b0 con 0x559142103ed0 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2709600640 10 client.15981 handle_command_reply: tid=0 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 client.15981 shutdown 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 msgr2=0x559142091e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2710006010 tx=0x7f2710102fe0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 shutdown_connections 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559142103ed0 0x5591421062c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559142089850 0x559142093320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591420846e0 0x559142092380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 --2- 172.21.15.90:0/1462318077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559142083cf0 0x559142091e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 >> 172.21.15.90:0/1462318077 conn(0x55914207f360 msgr2=0x559142085180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.144+0000 7f2709600640 0 client.15981 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.145+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 shutdown_connections 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.145+0000 7f2729b90740 1 -- 172.21.15.90:0/1462318077 wait complete. 2024-06-02T15:52:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.145+0000 7f2729b90740 20 client.15981 trim_cache size 0 max 16384 2024-06-02T15:52:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:52:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:52:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:52:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.146+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 msgr2=0x7f27040b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.146+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 0x7f27040b4e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x559142084f40 tx=0x7f2718009500 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.146+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 msgr2=0x7f272019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.146+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2710009a50 tx=0x7f27100042f0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 shutdown_connections 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f27040b29b0 0x7f27040b4e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f272010eeb0 0x7f27201a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2720076a90 0x7f272019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 --2- 172.21.15.90:0/2677075685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27200754b0 0x7f272019f160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 >> 172.21.15.90:0/2677075685 conn(0x7f27200fde50 msgr2=0x7f2720102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 shutdown_connections 2024-06-02T15:52:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:52:49.147+0000 7f2725600640 1 -- 172.21.15.90:0/2677075685 wait complete. 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.340+0000 7ff986600700 1 -- 172.21.15.90:0/965827154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9801035f0 msgr2=0x7ff980103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.340+0000 7ff986600700 1 --2- 172.21.15.90:0/965827154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9801035f0 0x7ff980103a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff968009990 tx=0x7ff968009ca0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 -- 172.21.15.90:0/965827154 shutdown_connections 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 --2- 172.21.15.90:0/965827154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff980103f90 0x7ff980110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 --2- 172.21.15.90:0/965827154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9801088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 --2- 172.21.15.90:0/965827154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9801035f0 0x7ff980103a50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 -- 172.21.15.90:0/965827154 >> 172.21.15.90:0/965827154 conn(0x7ff98007a6a0 msgr2=0x7ff98007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 -- 172.21.15.90:0/965827154 shutdown_connections 2024-06-02T15:52:49.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.341+0000 7ff986600700 1 -- 172.21.15.90:0/965827154 wait complete. 2024-06-02T15:52:49.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.342+0000 7ff986600700 1 Processor -- start 2024-06-02T15:52:49.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 -- start start 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 0x7ff980075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 0x7ff980075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113780 con 0x7ff980103f90 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113600 con 0x7ff9801035f0 2024-06-02T15:52:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.343+0000 7ff986600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff980113900 con 0x7ff980108500 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.344+0000 7ff97f400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 0x7ff980075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.344+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.344+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55384/0 (socket says 172.21.15.90:55384) 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.344+0000 7ff97fe00700 1 -- 172.21.15.90:0/2801193704 learned_addr learned my addr 172.21.15.90:0/2801193704 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.344+0000 7ff976a00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 0x7ff980075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 -- 172.21.15.90:0/2801193704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 msgr2=0x7ff980075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 0x7ff980075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 -- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 msgr2=0x7ff980075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 0x7ff980075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 -- 172.21.15.90:0/2801193704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801a8150 con 0x7ff980108500 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97f400700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 0x7ff980075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff97fe00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d6d0 tx=0x7ff97800d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.345+0000 7ff976a00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 0x7ff980075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.346+0000 7ff97cc00700 1 -- 172.21.15.90:0/2801193704 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978016070 con 0x7ff980108500 2024-06-02T15:52:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.346+0000 7ff97cc00700 1 -- 172.21.15.90:0/2801193704 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff978005020 con 0x7ff980108500 2024-06-02T15:52:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.346+0000 7ff97cc00700 1 -- 172.21.15.90:0/2801193704 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800fab0 con 0x7ff980108500 2024-06-02T15:52:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.346+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98006cfc0 con 0x7ff980108500 2024-06-02T15:52:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.346+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98006d4b0 con 0x7ff980108500 2024-06-02T15:52:49.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.350+0000 7ff97cc00700 1 -- 172.21.15.90:0/2801193704 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff97800fc70 con 0x7ff980108500 2024-06-02T15:52:49.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.352+0000 7ff97cc00700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 0x7ff9640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 Processor -- start 2024-06-02T15:52:49.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff97cc00700 1 -- 172.21.15.90:0/2801193704 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9780e2560 con 0x7ff980108500 2024-06-02T15:52:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 -- start start 2024-06-02T15:52:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6e610 0x558a2da6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558a2d9c6e40 con 0x558a2da69140 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a2d9b69e0 con 0x558a2da6e610 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.353+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a2d9baa20 con 0x558a2da6ef30 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.354+0000 7ff976a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.354+0000 7ff976a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55400/0 (socket says 172.21.15.90:55400) 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.354+0000 7ff976a00700 1 -- 172.21.15.90:0/3365871171 learned_addr learned my addr 172.21.15.90:0/3365871171 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.354+0000 7ff97f400700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 0x7ff9640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.354+0000 7ff97fe00700 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 -- 172.21.15.90:0/3365871171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6e610 msgr2=0x558a2da6e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6e610 0x558a2da6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 -- 172.21.15.90:0/3365871171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 msgr2=0x558a2da695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 -- 172.21.15.90:0/3365871171 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2d97f420 con 0x558a2da6ef30 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff976a00700 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff968009990 tx=0x7ff968009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=525fa2a23fdf8591 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.355+0000 7ff974c00700 1 -- 172.21.15.90:0/3365871171 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968038070 con 0x558a2da6ef30 2024-06-02T15:52:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.356+0000 7ff974c00700 1 -- 172.21.15.90:0/3365871171 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff968005070 con 0x558a2da6ef30 2024-06-02T15:52:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.356+0000 7ff974c00700 1 -- 172.21.15.90:0/3365871171 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ba30 con 0x558a2da6ef30 2024-06-02T15:52:49.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:49 smithi090 ceph-mon[31595]: pgmap v765: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 959 KiB/s wr, 204 op/s 2024-06-02T15:52:49.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff97f400700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 0x7ff9640b4dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff970009fd0 tx=0x7ff97000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3365871171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 msgr2=0x558a2da68b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff968009990 tx=0x7ff968009ee0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3365871171 shutdown_connections 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6e610 0x558a2da6e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3365871171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6ef30 0x558a2da68b20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.357+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3365871171 >> 172.21.15.90:0/3365871171 conn(0x558a2d996be0 msgr2=0x558a2da63690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.358+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3365871171 shutdown_connections 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.358+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3365871171 wait complete. 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.358+0000 7ff990f6ab80 1 -- start start 2024-06-02T15:52:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.358+0000 7ff990f6ab80 1 Processor -- start 2024-06-02T15:52:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da71f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558a2d8bcfa0 con 0x558a2da69140 2024-06-02T15:52:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x558a2d957760 con 0x558a2da6ef30 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a2da75980 con 0x558a2da6e610 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff990f6ab80 10 client.0 _ll_register_callbacks cb 0x558a2d960c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff97f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff97f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46424/0 (socket says 172.21.15.90:46424) 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.359+0000 7ff97f400700 1 -- 172.21.15.90:0/2355353300 learned_addr learned my addr 172.21.15.90:0/2355353300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da71f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 -- 172.21.15.90:0/2355353300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 msgr2=0x558a2da724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 -- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 msgr2=0x558a2da74bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 -- 172.21.15.90:0/2355353300 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2d9068d0 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff97f400700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.360+0000 7ff976a00700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da71f90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff968006010 tx=0x7ff968005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fad4e82b746d18c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff96d600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968038040 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff968003860 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff968005070 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff96804a050 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff990f6ab80 10 client.26191 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558a2d987060 con 0x558a2da6e610 2024-06-02T15:52:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.361+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff968005070 con 0x558a2da6e610 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ff968033480 con 0x558a2da6e610 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 10 client.26191 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 10 client.26191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 20 client.26191 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2da822e0 0x558a2da846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 4 client.26191 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.362+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a2d9c8d90 con 0x558a2da822e0 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.363+0000 7ff97f400700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2da822e0 0x558a2da846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.363+0000 7ff97f400700 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2da822e0 0x558a2da846d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a95c45b52c5f1df server_cookie=dea4f0c342e517e1 in_seq=0 out_seq=0 2024-06-02T15:52:49.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.364+0000 7ff96d600700 10 client.26191 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:52:49.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.366+0000 7ff96d600700 1 -- 172.21.15.90:0/2355353300 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558a2d9c8d90 con 0x558a2da822e0 2024-06-02T15:52:49.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.366+0000 7ff96d600700 10 client.26191 handle_command_reply: tid=0 2024-06-02T15:52:49.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.366+0000 7ff990f6ab80 1 client.26191 shutdown 2024-06-02T15:52:49.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.366+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 msgr2=0x558a2da71f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.366+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da71f90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff968006010 tx=0x7ff968005500 comp rx=0 tx=0).stop 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 shutdown_connections 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2da822e0 0x558a2da846d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da74bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/2355353300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da71f90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 >> 172.21.15.90:0/2355353300 conn(0x558a2d996be0 msgr2=0x558a2da62d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff96d600700 0 client.26191 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:52:49.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 shutdown_connections 2024-06-02T15:52:49.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 1 -- 172.21.15.90:0/2355353300 wait complete. 2024-06-02T15:52:49.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.367+0000 7ff990f6ab80 20 client.26191 trim_cache size 0 max 16384 2024-06-02T15:52:49.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.389+0000 7ff990f6ab80 1 Processor -- start 2024-06-02T15:52:49.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.389+0000 7ff990f6ab80 1 -- start start 2024-06-02T15:52:49.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da69140 0x558a2da68b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801ab650 con 0x558a2da6ef30 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801a8e20 con 0x558a2da69140 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800740b0 con 0x558a2da6e610 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff97f400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da69140 0x558a2da68b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff976a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46426/0 (socket says 172.21.15.90:46426) 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.390+0000 7ff97fe00700 1 -- 172.21.15.90:0/3379240625 learned_addr learned my addr 172.21.15.90:0/3379240625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 -- 172.21.15.90:0/3379240625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da69140 msgr2=0x558a2da68b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da69140 0x558a2da68b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 -- 172.21.15.90:0/3379240625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 msgr2=0x558a2da6f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 -- 172.21.15.90:0/3379240625 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2da70500 con 0x558a2da6e610 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff97fe00700 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.391+0000 7ff976a00700 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da634f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff968033a40 tx=0x7ff968009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5412fe6c0616c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.392+0000 7ff96fe00700 1 -- 172.21.15.90:0/3379240625 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968033480 con 0x558a2da6e610 2024-06-02T15:52:49.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.392+0000 7ff96fe00700 1 -- 172.21.15.90:0/3379240625 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96802c030 con 0x558a2da6e610 2024-06-02T15:52:49.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.392+0000 7ff96fe00700 1 -- 172.21.15.90:0/3379240625 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ba30 con 0x558a2da6e610 2024-06-02T15:52:49.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.392+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3379240625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 msgr2=0x558a2da634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.392+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da634f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff968033a40 tx=0x7ff968009f60 comp rx=0 tx=0).stop 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.393+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3379240625 shutdown_connections 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.393+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da6ef30 0x558a2da6f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.393+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da69140 0x558a2da68b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.393+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/3379240625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da6e610 0x558a2da634f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.393+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3379240625 >> 172.21.15.90:0/3379240625 conn(0x558a2d996be0 msgr2=0x558a2da71f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.394+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3379240625 shutdown_connections 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.394+0000 7ff990f6ab80 1 -- 172.21.15.90:0/3379240625 wait complete. 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.394+0000 7ff990f6ab80 1 -- start start 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 Processor -- start 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff980074720 con 0x558a2da69140 2024-06-02T15:52:49.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801a8750 con 0x558a2da6ef30 2024-06-02T15:52:49.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff980074df0 con 0x558a2da858c0 2024-06-02T15:52:49.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.395+0000 7ff990f6ab80 10 client.0 _ll_register_callbacks cb 0x558a2d960c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:52:49.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.396+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.396+0000 7ff97fe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:55434/0 (socket says 172.21.15.90:55434) 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.396+0000 7ff97fe00700 1 -- 172.21.15.90:0/1918971257 learned_addr learned my addr 172.21.15.90:0/1918971257 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff976a00700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97f400700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 -- 172.21.15.90:0/1918971257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 msgr2=0x558a2da85270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da85270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 -- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 msgr2=0x558a2da87f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da87f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 -- 172.21.15.90:0/1918971257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2daf45e0 con 0x558a2da858c0 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff97fe00700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d6a0 tx=0x7ff9780a9330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66abcf7698d54d50 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:52:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff96d600700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.397+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9780adb20 con 0x558a2da858c0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.398+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7ff9780e2220 con 0x558a2da858c0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.398+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff96802c030 con 0x558a2da858c0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.398+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff9780b25d0 con 0x558a2da858c0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.398+0000 7ff990f6ab80 10 client.26197 fetch_fsmap learned FSMap version 37 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.398+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558a2daf5830 con 0x558a2da858c0 2024-06-02T15:52:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780c7020 con 0x558a2da858c0 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ff97808fce0 con 0x558a2da858c0 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 10 client.26197 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 10 client.26197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 20 client.26197 populate_metadata read hostname 'smithi090' 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2daf5f60 0x558a2daf8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 4 client.26197 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.399+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a2d9c2570 con 0x558a2daf5f60 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.400+0000 7ff97f400700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2daf5f60 0x558a2daf8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.401+0000 7ff97f400700 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2daf5f60 0x558a2daf8380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be31ef4407e20796 server_cookie=832580ff19dc18c7 in_seq=0 out_seq=0 2024-06-02T15:52:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.401+0000 7ff96d600700 10 client.26197 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:52:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.404+0000 7ff96d600700 1 -- 172.21.15.90:0/1918971257 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a2d9c2570 con 0x558a2daf5f60 2024-06-02T15:52:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.404+0000 7ff96d600700 10 client.26197 handle_command_reply: tid=0 2024-06-02T15:52:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.404+0000 7ff990f6ab80 1 client.26197 shutdown 2024-06-02T15:52:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.404+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 msgr2=0x558a2da85d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.404+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d6a0 tx=0x7ff9780a9330 comp rx=0 tx=0).stop 2024-06-02T15:52:49.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 shutdown_connections 2024-06-02T15:52:49.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x558a2daf5f60 0x558a2daf8380 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a2da858c0 0x558a2da85d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a2da69140 0x558a2da87f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 --2- 172.21.15.90:0/1918971257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x558a2da6ef30 0x558a2da85270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 >> 172.21.15.90:0/1918971257 conn(0x558a2d996be0 msgr2=0x558a2da6f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.405+0000 7ff96d600700 0 client.26197 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:52:49.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.407+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 shutdown_connections 2024-06-02T15:52:49.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.407+0000 7ff990f6ab80 1 -- 172.21.15.90:0/1918971257 wait complete. 2024-06-02T15:52:49.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.407+0000 7ff990f6ab80 20 client.26197 trim_cache size 0 max 16384 2024-06-02T15:52:49.406 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:52:49.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.408+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 msgr2=0x7ff9640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.408+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 0x7ff9640b4dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff970009fd0 tx=0x7ff97000b040 comp rx=0 tx=0).stop 2024-06-02T15:52:49.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.408+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 msgr2=0x7ff9800762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.408+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff97800d6d0 tx=0x7ff97800d9e0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 shutdown_connections 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff9640b2900 0x7ff9640b4dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff980108500 0x7ff9800762c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9801035f0 0x7ff980075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 --2- 172.21.15.90:0/2801193704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff980103f90 0x7ff980075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:52:49.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.409+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 >> 172.21.15.90:0/2801193704 conn(0x7ff98007a6a0 msgr2=0x7ff980101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:52:49.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.410+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 shutdown_connections 2024-06-02T15:52:49.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:52:49.410+0000 7ff986600700 1 -- 172.21.15.90:0/2801193704 wait complete. 2024-06-02T15:52:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:49 smithi120 ceph-mon[36451]: pgmap v765: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 959 KiB/s wr, 204 op/s 2024-06-02T15:52:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:49 smithi135 ceph-mon[36839]: pgmap v765: 129 pgs: 129 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 959 KiB/s wr, 204 op/s 2024-06-02T15:52:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:51 smithi120 ceph-mon[36451]: pgmap v766: 129 pgs: 129 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-06-02T15:52:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:51 smithi135 ceph-mon[36839]: pgmap v766: 129 pgs: 129 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-06-02T15:52:51.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:51 smithi090 ceph-mon[31595]: pgmap v766: 129 pgs: 129 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-06-02T15:52:53.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:52 smithi120 ceph-mon[36451]: pgmap v767: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-06-02T15:52:53.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:52 smithi135 ceph-mon[36839]: pgmap v767: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-06-02T15:52:53.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:52 smithi090 ceph-mon[31595]: pgmap v767: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-06-02T15:52:55.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:55 smithi120 ceph-mon[36451]: pgmap v768: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-06-02T15:52:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:55 smithi135 ceph-mon[36839]: pgmap v768: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-06-02T15:52:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:55 smithi090 ceph-mon[31595]: pgmap v768: 129 pgs: 129 active+clean; 254 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-06-02T15:52:56.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:52:56.833 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:52:57.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:57 smithi090 ceph-mon[31595]: pgmap v769: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-06-02T15:52:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:57 smithi120 ceph-mon[36451]: pgmap v769: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-06-02T15:52:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:57 smithi135 ceph-mon[36839]: pgmap v769: 129 pgs: 129 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-06-02T15:52:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:52:59 smithi120 ceph-mon[36451]: pgmap v770: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 223 op/s 2024-06-02T15:52:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:52:59 smithi135 ceph-mon[36839]: pgmap v770: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 223 op/s 2024-06-02T15:52:59.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:52:59 smithi090 ceph-mon[31595]: pgmap v770: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 223 op/s 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.103+0000 7fbde7200700 1 -- 172.21.15.90:0/3829655275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde0103f80 msgr2=0x7fbde0110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.103+0000 7fbde7200700 1 --2- 172.21.15.90:0/3829655275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde0103f80 0x7fbde0110410 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80098f0 tx=0x7fbdd8009c00 comp rx=0 tx=0).stop 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.104+0000 7fbde7200700 1 -- 172.21.15.90:0/3829655275 shutdown_connections 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.104+0000 7fbde7200700 1 --2- 172.21.15.90:0/3829655275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde0103f80 0x7fbde0110410 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.104+0000 7fbde7200700 1 --2- 172.21.15.90:0/3829655275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01095d0 0x7fbde01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.104+0000 7fbde7200700 1 --2- 172.21.15.90:0/3829655275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde01035c0 0x7fbde0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.104+0000 7fbde7200700 1 -- 172.21.15.90:0/3829655275 >> 172.21.15.90:0/3829655275 conn(0x7fbde007a610 msgr2=0x7fbde007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.105+0000 7fbde7200700 1 -- 172.21.15.90:0/3829655275 shutdown_connections 2024-06-02T15:53:01.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.105+0000 7fbde7200700 1 -- 172.21.15.90:0/3829655275 wait complete. 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.107+0000 7fbde7200700 1 Processor -- start 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.108+0000 7fbde7200700 1 -- start start 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbde7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01035c0 0x7fbde01a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbde7200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbde7200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 0x7fbde010d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57066/0 (socket says 172.21.15.90:57066) 2024-06-02T15:53:01.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbdd7400700 1 -- 172.21.15.90:0/1445355984 learned_addr learned my addr 172.21.15.90:0/1445355984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.109+0000 7fbde5200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 0x7fbde010d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.110+0000 7fbddfe00700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01035c0 0x7fbde01a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.110+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01137d0 con 0x7fbde01035c0 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.110+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0113650 con 0x7fbde01095d0 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.110+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0113950 con 0x7fbde0103f80 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.110+0000 7fbde5200700 1 -- 172.21.15.90:0/1445355984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 msgr2=0x7fbde01a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbde5200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbde5200700 1 -- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01035c0 msgr2=0x7fbde01a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbde5200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01035c0 0x7fbde01a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbde5200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde010dbe0 con 0x7fbde01095d0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbdd7400700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.111+0000 7fbde5200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 0x7fbde010d420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009810 tx=0x7fbdd8009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.112+0000 7fbddd600700 1 -- 172.21.15.90:0/1445355984 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8027070 con 0x7fbde01095d0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.112+0000 7fbddd600700 1 -- 172.21.15.90:0/1445355984 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd800b560 con 0x7fbde01095d0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.112+0000 7fbddd600700 1 -- 172.21.15.90:0/1445355984 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80049c0 con 0x7fbde01095d0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.112+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde010de70 con 0x7fbde01095d0 2024-06-02T15:53:01.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.112+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01ace40 con 0x7fbde01095d0 2024-06-02T15:53:01.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.118+0000 7fbddd600700 1 -- 172.21.15.90:0/1445355984 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbdd802a070 con 0x7fbde01095d0 2024-06-02T15:53:01.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.121+0000 7fbdf1c23b80 1 Processor -- start 2024-06-02T15:53:01.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.126+0000 7fbddd600700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 0x7fbdc40b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.127+0000 7fbddd600700 1 -- 172.21.15.90:0/1445355984 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbdd80e2db0 con 0x7fbde01095d0 2024-06-02T15:53:01.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.127+0000 7fbdf1c23b80 1 -- start start 2024-06-02T15:53:01.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.128+0000 7fbdd7400700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 0x7fbdc40b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.128+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.128+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965d91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.129+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d91880 0x55f965d91ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.129+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f965cefe40 con 0x55f965d91880 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.129+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f965cdf9e0 con 0x55f965d977d0 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.129+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f965ce3a20 con 0x55f965d96eb0 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.129+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.130+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57070/0 (socket says 172.21.15.90:57070) 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.130+0000 7fbddfe00700 1 -- 172.21.15.90:0/1801347752 learned_addr learned my addr 172.21.15.90:0/1801347752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbdd7400700 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965d91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbddfe00700 1 -- 172.21.15.90:0/1801347752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 msgr2=0x55f965d91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbddfe00700 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965d91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbddfe00700 1 -- 172.21.15.90:0/1801347752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d91880 msgr2=0x55f965d91ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbddfe00700 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d91880 0x55f965d91ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbddfe00700 1 -- 172.21.15.90:0/1801347752 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f965ca8420 con 0x55f965d96eb0 2024-06-02T15:53:01.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbdd7400700 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965d91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:01.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.131+0000 7fbdd7400700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 0x7fbdc40b4f10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009750 tx=0x7fbdc8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.132+0000 7fbddfe00700 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000a4b0 tx=0x7fbdd00096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dde2dbe33a2c964 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.133+0000 7fbdd5600700 1 -- 172.21.15.90:0/1801347752 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0013790 con 0x55f965d96eb0 2024-06-02T15:53:01.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.133+0000 7fbdd5600700 1 -- 172.21.15.90:0/1801347752 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0019070 con 0x55f965d96eb0 2024-06-02T15:53:01.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.133+0000 7fbdd5600700 1 -- 172.21.15.90:0/1801347752 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0014bf0 con 0x55f965d96eb0 2024-06-02T15:53:01.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.134+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/1801347752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 msgr2=0x55f965d97290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.134+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000a4b0 tx=0x7fbdd00096d0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.135+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/1801347752 shutdown_connections 2024-06-02T15:53:01.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.135+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d91880 0x55f965d91ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.135+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d97290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.135+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/1801347752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965d91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.135+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/1801347752 >> 172.21.15.90:0/1801347752 conn(0x55f965cbfbe0 msgr2=0x55f965d8c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.136+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/1801347752 shutdown_connections 2024-06-02T15:53:01.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.136+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/1801347752 wait complete. 2024-06-02T15:53:01.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.137+0000 7fbdf1c23b80 1 -- start start 2024-06-02T15:53:01.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.137+0000 7fbdf1c23b80 1 Processor -- start 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d96420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965d93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f965be5fa0 con 0x55f965d96eb0 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f965c80760 con 0x55f965d91880 2024-06-02T15:53:01.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f965daf250 con 0x55f965d977d0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.138+0000 7fbdf1c23b80 10 client.0 _ll_register_callbacks cb 0x55f965c89c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbde5200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbde5200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57086/0 (socket says 172.21.15.90:57086) 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d96420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d96420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42548/0 (socket says 172.21.15.90:42548) 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbddfe00700 1 -- 172.21.15.90:0/4202162285 learned_addr learned my addr 172.21.15.90:0/4202162285 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.140+0000 7fbdd7400700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965d93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 -- 172.21.15.90:0/4202162285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 msgr2=0x55f965d96420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d96420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 -- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 msgr2=0x55f965d93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965d93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 -- 172.21.15.90:0/4202162285 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f965c2f8d0 con 0x55f965d977d0 2024-06-02T15:53:01.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.141+0000 7fbde5200700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c010 tx=0x7fbdd80aa5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5570e3b4f728cf8f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdce000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80a9b60 con 0x55f965d977d0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbdd8004d70 con 0x55f965d977d0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd0019070 con 0x55f965d977d0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbdd80aa750 con 0x55f965d977d0 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdf1c23b80 10 client.26209 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:01.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.151+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f965cb0060 con 0x55f965d977d0 2024-06-02T15:53:01.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.152+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0019070 con 0x55f965d977d0 2024-06-02T15:53:01.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fbdd80a59c0 con 0x55f965d977d0 2024-06-02T15:53:01.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:01.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 10 client.26209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:53:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 20 client.26209 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965dcf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 4 client.26209 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.153+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f965cf1d90 con 0x55f965dcce20 2024-06-02T15:53:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.154+0000 7fbddfe00700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965dcf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.158+0000 7fbddfe00700 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965dcf240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c1603830229bd63 server_cookie=82bb27820f534338 in_seq=0 out_seq=0 2024-06-02T15:53:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.158+0000 7fbdce000700 10 client.26209 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:53:01.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.165+0000 7fbdce000700 1 -- 172.21.15.90:0/4202162285 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f965cf1d90 con 0x55f965dcce20 2024-06-02T15:53:01.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.165+0000 7fbdce000700 10 client.26209 handle_command_reply: tid=0 2024-06-02T15:53:01.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.165+0000 7fbdf1c23b80 1 client.26209 shutdown 2024-06-02T15:53:01.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.166+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 msgr2=0x55f965d93d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.166+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800c010 tx=0x7fbdd80aa5b0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.166+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 shutdown_connections 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.166+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d977d0 0x55f965d93d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.166+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965dcf240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.167+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d96420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.167+0000 7fbdce000700 0 client.26209 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.167+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/4202162285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965d93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.167+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 >> 172.21.15.90:0/4202162285 conn(0x55f965cbfbe0 msgr2=0x55f965d8e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.168+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 shutdown_connections 2024-06-02T15:53:01.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.168+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/4202162285 wait complete. 2024-06-02T15:53:01.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.168+0000 7fbdf1c23b80 20 client.26209 trim_cache size 0 max 16384 2024-06-02T15:53:01.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.200+0000 7fbdf1c23b80 1 Processor -- start 2024-06-02T15:53:01.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.200+0000 7fbdf1c23b80 1 -- start start 2024-06-02T15:53:01.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 0x55f965d93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57094/0 (socket says 172.21.15.90:57094) 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.201+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.202+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42556/0 (socket says 172.21.15.90:42556) 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.202+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc0115a0 con 0x55f965d977d0 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.202+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00ed70 con 0x55f965d91880 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.202+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00cbc0 con 0x55f965d96eb0 2024-06-02T15:53:01.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.203+0000 7fbdd7400700 1 -- 172.21.15.90:0/2161283540 learned_addr learned my addr 172.21.15.90:0/2161283540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.203+0000 7fbde5200700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 0x55f965d93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbdd7400700 1 -- 172.21.15.90:0/2161283540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 msgr2=0x55f965d91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbdd7400700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbdd7400700 1 -- 172.21.15.90:0/2161283540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 msgr2=0x55f965d93b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbddfe00700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-06-02T15:53:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbdd7400700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 0x55f965d93b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbdd7400700 1 -- 172.21.15.90:0/2161283540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f965daf130 con 0x55f965d96eb0 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.204+0000 7fbde5200700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 0x55f965d93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.205+0000 7fbdd7400700 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005d10 tx=0x7fbdc8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35632f3730120a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.205+0000 7fbdd4c00700 1 -- 172.21.15.90:0/2161283540 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8029440 con 0x55f965d96eb0 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.205+0000 7fbdd4c00700 1 -- 172.21.15.90:0/2161283540 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc8029a90 con 0x55f965d96eb0 2024-06-02T15:53:01.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.205+0000 7fbdd4c00700 1 -- 172.21.15.90:0/2161283540 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc801f950 con 0x55f965d96eb0 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/2161283540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 msgr2=0x55f965d935f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8005d10 tx=0x7fbdc8014000 comp rx=0 tx=0).stop 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/2161283540 shutdown_connections 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d977d0 0x55f965d93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d91880 0x55f965d91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/2161283540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d96eb0 0x55f965d935f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.206+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/2161283540 >> 172.21.15.90:0/2161283540 conn(0x55f965cbfbe0 msgr2=0x55f965d85e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.208+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/2161283540 shutdown_connections 2024-06-02T15:53:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.208+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/2161283540 wait complete. 2024-06-02T15:53:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.209+0000 7fbdf1c23b80 1 -- start start 2024-06-02T15:53:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.209+0000 7fbdf1c23b80 1 Processor -- start 2024-06-02T15:53:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.210+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 0x55f965e2fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.210+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.210+0000 7fbdf1c23b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965db0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.211+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00d230 con 0x55f965d96eb0 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.211+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00e6a0 con 0x55f965d977d0 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.211+0000 7fbdf1c23b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00d900 con 0x55f965d91880 2024-06-02T15:53:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.211+0000 7fbde5200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965db0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.214+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.214+0000 7fbdd7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44386/0 (socket says 172.21.15.90:44386) 2024-06-02T15:53:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.214+0000 7fbdd7400700 1 -- 172.21.15.90:0/183204708 learned_addr learned my addr 172.21.15.90:0/183204708 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.214+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 0x55f965e2fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.215+0000 7fbddfe00700 1 -- 172.21.15.90:0/183204708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 msgr2=0x55f965db0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.215+0000 7fbddfe00700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965db0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.215+0000 7fbdf1c23b80 10 client.0 _ll_register_callbacks cb 0x55f965c89c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:01.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.215+0000 7fbddfe00700 1 -- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 msgr2=0x55f965e30230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:01.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbddfe00700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbddfe00700 1 -- 172.21.15.90:0/183204708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f965db12c0 con 0x55f965d91880 2024-06-02T15:53:01.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbde5200700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965db0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:01.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbdd7400700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:01.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc8029a90 con 0x55f965d91880 2024-06-02T15:53:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.216+0000 7fbddfe00700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 0x55f965e2fcf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0014790 tx=0x7fbdd0009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef83eba7bde0004 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.217+0000 7fbdce000700 10 client.26215 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:53:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.217+0000 7fbdce000700 1 -- 172.21.15.90:0/183204708 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0014ca0 con 0x55f965d91880 2024-06-02T15:53:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.217+0000 7fbdce000700 1 -- 172.21.15.90:0/183204708 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fbdd00273b0 con 0x55f965d91880 2024-06-02T15:53:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.217+0000 7fbdce000700 1 -- 172.21.15.90:0/183204708 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbdd0027650 con 0x55f965d91880 2024-06-02T15:53:01.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.218+0000 7fbdf1c23b80 10 client.26215 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:01.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.218+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55f965e38470 con 0x55f965d91880 2024-06-02T15:53:01.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.219+0000 7fbdce000700 1 -- 172.21.15.90:0/183204708 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fbdd00277f0 con 0x55f965d91880 2024-06-02T15:53:01.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.219+0000 7fbdf1c23b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:01.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.219+0000 7fbdf1c23b80 10 client.26215 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:53:01.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.219+0000 7fbdf1c23b80 20 client.26215 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.220+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965e3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.220+0000 7fbdf1c23b80 4 client.26215 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:53:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.220+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f965ceb570 con 0x55f965dcce20 2024-06-02T15:53:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.220+0000 7fbdd7400700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965e3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.221+0000 7fbdce000700 10 client.26215 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:53:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.221+0000 7fbdd7400700 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965e3a8d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65ddbfa0eb2a7846 server_cookie=c268a7e3c02d4a2a in_seq=0 out_seq=0 2024-06-02T15:53:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.229+0000 7fbdce000700 1 -- 172.21.15.90:0/183204708 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f965ceb570 con 0x55f965dcce20 2024-06-02T15:53:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.229+0000 7fbdce000700 10 client.26215 handle_command_reply: tid=0 2024-06-02T15:53:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.229+0000 7fbdf1c23b80 1 client.26215 shutdown 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 msgr2=0x55f965e2fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 0x55f965e2fcf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0014790 tx=0x7fbdd0009c70 comp rx=0 tx=0).stop 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 shutdown_connections 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f965dcce20 0x55f965e3a8d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f965d977d0 0x55f965db0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f965d91880 0x55f965e2fcf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 --2- 172.21.15.90:0/183204708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f965d96eb0 0x55f965e30230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 >> 172.21.15.90:0/183204708 conn(0x55f965cbfbe0 msgr2=0x55f965d97c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.230+0000 7fbdce000700 0 client.26215 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:53:01.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.231+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 shutdown_connections 2024-06-02T15:53:01.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.231+0000 7fbdf1c23b80 1 -- 172.21.15.90:0/183204708 wait complete. 2024-06-02T15:53:01.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.231+0000 7fbdf1c23b80 20 client.26215 trim_cache size 0 max 16384 2024-06-02T15:53:01.228 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.232+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 msgr2=0x7fbdc40b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.232+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 0x7fbdc40b4f10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009750 tx=0x7fbdc8009310 comp rx=0 tx=0).stop 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.232+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 msgr2=0x7fbde010d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.232+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 0x7fbde010d420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009810 tx=0x7fbdd8009480 comp rx=0 tx=0).stop 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 shutdown_connections 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdc40b2a50 0x7fbdc40b4f10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbde01095d0 0x7fbde010d420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde01035c0 0x7fbde01a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 --2- 172.21.15.90:0/1445355984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbde0103f80 0x7fbde01a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:01.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 >> 172.21.15.90:0/1445355984 conn(0x7fbde007a610 msgr2=0x7fbde010e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:01.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 shutdown_connections 2024-06-02T15:53:01.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:01.233+0000 7fbde7200700 1 -- 172.21.15.90:0/1445355984 wait complete. 2024-06-02T15:53:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:01 smithi120 ceph-mon[36451]: pgmap v771: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 232 op/s 2024-06-02T15:53:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:01 smithi135 ceph-mon[36839]: pgmap v771: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 232 op/s 2024-06-02T15:53:01.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:01 smithi090 ceph-mon[31595]: pgmap v771: 129 pgs: 129 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 232 op/s 2024-06-02T15:53:02.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:53:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 -- 172.21.15.90:0/1762906276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5880103570 msgr2=0x7f5880105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 --2- 172.21.15.90:0/1762906276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5880103570 0x7f5880105960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5874009900 tx=0x7f587402a400 comp rx=0 tx=0).stop 2024-06-02T15:53:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 -- 172.21.15.90:0/1762906276 shutdown_connections 2024-06-02T15:53:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 --2- 172.21.15.90:0/1762906276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58801087d0 0x7f588010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 --2- 172.21.15.90:0/1762906276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880105ea0 0x7f5880108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 --2- 172.21.15.90:0/1762906276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5880103570 0x7f5880105960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.391+0000 7f5888000640 1 -- 172.21.15.90:0/1762906276 >> 172.21.15.90:0/1762906276 conn(0x7f58800fcf20 msgr2=0x7f58800ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.392+0000 7f5888000640 1 -- 172.21.15.90:0/1762906276 shutdown_connections 2024-06-02T15:53:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.393+0000 7f5888000640 1 -- 172.21.15.90:0/1762906276 wait complete. 2024-06-02T15:53:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.395+0000 7f5888000640 1 Processor -- start 2024-06-02T15:53:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.396+0000 7f5888000640 1 -- start start 2024-06-02T15:53:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.398+0000 7f5888000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 0x7f5880102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.398+0000 7f5888000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 0x7f5880101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.398+0000 7f5885600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 0x7f5880102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.398+0000 7f5885600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 0x7f5880102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57126/0 (socket says 172.21.15.90:57126) 2024-06-02T15:53:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.398+0000 7f5885600640 1 -- 172.21.15.90:0/735612672 learned_addr learned my addr 172.21.15.90:0/735612672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.399+0000 7f5884c00640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 0x7f5880101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.399+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58801087d0 0x7f5880101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.399+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f588010e5d0 con 0x7f58801087d0 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.399+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f588010e450 con 0x7f5880105ea0 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.399+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f588010e750 con 0x7f5880103570 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5886000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58801087d0 0x7f5880101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5884c00640 1 -- 172.21.15.90:0/735612672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 msgr2=0x7f5880102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5884c00640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 0x7f5880102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5884c00640 1 -- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58801087d0 msgr2=0x7f5880101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5884c00640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58801087d0 0x7f5880101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.400+0000 7f5884c00640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58801022f0 con 0x7f5880105ea0 2024-06-02T15:53:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.401+0000 7f5884c00640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 0x7f5880101520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f587000c940 tx=0x7f587000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.402+0000 7f587e000640 1 -- 172.21.15.90:0/735612672 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870007bb0 con 0x7f5880105ea0 2024-06-02T15:53:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.402+0000 7f587e000640 1 -- 172.21.15.90:0/735612672 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5870007d50 con 0x7f5880105ea0 2024-06-02T15:53:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.402+0000 7f587e000640 1 -- 172.21.15.90:0/735612672 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870018730 con 0x7f5880105ea0 2024-06-02T15:53:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.405+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58801a2d20 con 0x7f5880105ea0 2024-06-02T15:53:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.405+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a3190 con 0x7f5880105ea0 2024-06-02T15:53:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.409+0000 7f588ada5740 1 Processor -- start 2024-06-02T15:53:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.412+0000 7f588ada5740 1 -- start start 2024-06-02T15:53:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.412+0000 7f588ada5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f588ada5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f588ada5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f588ada5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557d0e148b90 con 0x557d0e1de690 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f588ada5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d0e138730 con 0x557d0e1d8b30 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f588ada5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557d0e13c770 con 0x557d0e1d9520 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f5886000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f5886000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42596/0 (socket says 172.21.15.90:42596) 2024-06-02T15:53:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f5886000640 1 -- 172.21.15.90:0/2941801915 learned_addr learned my addr 172.21.15.90:0/2941801915 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f587e000640 1 -- 172.21.15.90:0/735612672 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f58700188d0 con 0x7f5880105ea0 2024-06-02T15:53:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.413+0000 7f5885600640 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.415+0000 7f587e000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 0x7f58680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.416+0000 7f587e000640 1 -- 172.21.15.90:0/735612672 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f58700e24e0 con 0x7f5880105ea0 2024-06-02T15:53:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.417+0000 7f5886000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 0x7f58680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.417+0000 7f5886000640 1 -- 172.21.15.90:0/2941801915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 msgr2=0x557d0e1de020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.418+0000 7f5886000640 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1de020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.418+0000 7f5886000640 1 -- 172.21.15.90:0/2941801915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 msgr2=0x557d0e1e0b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.418+0000 7f5886000640 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1e0b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.418+0000 7f5886000640 1 -- 172.21.15.90:0/2941801915 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d0e0d18b0 con 0x557d0e1d8b30 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.418+0000 7f5885600640 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.419+0000 7f5886000640 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5878015dc0 tx=0x7f58780127c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83b2a1c07aa50da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.419+0000 7f5886000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 0x7f58680b4f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5878012cc0 tx=0x7f587801c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.420+0000 7f586e600640 1 -- 172.21.15.90:0/2941801915 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878022070 con 0x557d0e1d8b30 2024-06-02T15:53:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.420+0000 7f586e600640 1 -- 172.21.15.90:0/2941801915 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878010c60 con 0x557d0e1d8b30 2024-06-02T15:53:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 -- 172.21.15.90:0/2941801915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 msgr2=0x557d0e1d8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5878015dc0 tx=0x7f58780127c0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 -- 172.21.15.90:0/2941801915 shutdown_connections 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1de020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 --2- 172.21.15.90:0/2941801915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1d8f50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 -- 172.21.15.90:0/2941801915 >> 172.21.15.90:0/2941801915 conn(0x557d0e1d41a0 msgr2=0x557d0e1d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 -- 172.21.15.90:0/2941801915 shutdown_connections 2024-06-02T15:53:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.421+0000 7f588ada5740 1 -- 172.21.15.90:0/2941801915 wait complete. 2024-06-02T15:53:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.422+0000 7f588ada5740 1 -- start start 2024-06-02T15:53:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.422+0000 7f588ada5740 1 Processor -- start 2024-06-02T15:53:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 0x557d0e1eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1de690 0x557d0e1f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557d0de1e790 con 0x557d0e1d8b30 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557d0e0e0bc0 con 0x557d0e1d9520 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557d0e1e1680 con 0x557d0e1de690 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.423+0000 7f588ada5740 10 client.0 _ll_register_callbacks cb 0x557d0e0be400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.424+0000 7f5886000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1de690 0x557d0e1f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42606/0 (socket says 172.21.15.90:42606) 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 -- 172.21.15.90:0/1086373777 learned_addr learned my addr 172.21.15.90:0/1086373777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5885600640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 0x557d0e1eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 -- 172.21.15.90:0/1086373777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1de690 msgr2=0x557d0e1f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1de690 0x557d0e1f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 -- 172.21.15.90:0/1086373777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 msgr2=0x557d0e1eb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 0x557d0e1eb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5884c00640 1 -- 172.21.15.90:0/1086373777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d0de4d210 con 0x557d0e1d9520 2024-06-02T15:53:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.425+0000 7f5885600640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 0x557d0e1eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.426+0000 7f5884c00640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58700a9470 tx=0x7f58700a9d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70dd9f5608d1ef5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.426+0000 7f5862a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.426+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700c2070 con 0x557d0e1d9520 2024-06-02T15:53:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.427+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5878010c60 con 0x557d0e1d9520 2024-06-02T15:53:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.427+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f58700a5d60 con 0x557d0e1d9520 2024-06-02T15:53:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.427+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f58700c2070 con 0x557d0e1d9520 2024-06-02T15:53:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.427+0000 7f588ada5740 10 client.26294 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.427+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d0df19620 con 0x557d0e1d9520 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878010c60 con 0x557d0e1d9520 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5870020020 con 0x557d0e1d9520 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f588ada5740 10 client.26294 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f588ada5740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f588ada5740 20 client.26294 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.428+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e1e65c0 0x557d0e1e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.429+0000 7f588ada5740 4 client.26294 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.429+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d0e14aae0 con 0x557d0e1e65c0 2024-06-02T15:53:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.430+0000 7f5885600640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e1e65c0 0x557d0e1e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.431+0000 7f5885600640 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e1e65c0 0x557d0e1e89e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd2055dd7787daa server_cookie=ce7fa60582de3265 in_seq=0 out_seq=0 2024-06-02T15:53:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.431+0000 7f5862a00640 10 client.26294 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.432+0000 7f5862a00640 1 -- 172.21.15.90:0/1086373777 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557d0e14aae0 con 0x557d0e1e65c0 2024-06-02T15:53:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.432+0000 7f5862a00640 10 client.26294 handle_command_reply: tid=0 2024-06-02T15:53:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.432+0000 7f588ada5740 1 client.26294 shutdown 2024-06-02T15:53:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 msgr2=0x557d0e1eb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f58700a9470 tx=0x7f58700a9d90 comp rx=0 tx=0).stop 2024-06-02T15:53:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 shutdown_connections 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e1e65c0 0x557d0e1e89e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1de690 0x557d0e1f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f5862a00640 0 client.26294 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d9520 0x557d0e1eb610 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 --2- 172.21.15.90:0/1086373777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1d8b30 0x557d0e1eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 >> 172.21.15.90:0/1086373777 conn(0x557d0e1d41a0 msgr2=0x557d0e1d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 shutdown_connections 2024-06-02T15:53:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.433+0000 7f588ada5740 1 -- 172.21.15.90:0/1086373777 wait complete. 2024-06-02T15:53:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.434+0000 7f588ada5740 20 client.26294 trim_cache size 0 max 16384 2024-06-02T15:53:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 Processor -- start 2024-06-02T15:53:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 -- start start 2024-06-02T15:53:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58801abd00 con 0x557d0e1de690 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a9510 con 0x557d0e1d8b30 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.465+0000 7f588ada5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a7360 con 0x557d0e1d9520 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5885600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5885600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1da630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42612/0 (socket says 172.21.15.90:42612) 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5885600640 1 -- 172.21.15.90:0/511674401 learned_addr learned my addr 172.21.15.90:0/511674401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 -- 172.21.15.90:0/511674401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 msgr2=0x557d0e1da630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1da630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 -- 172.21.15.90:0/511674401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 msgr2=0x557d0e1d2a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1d2a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.466+0000 7f5884c00640 1 -- 172.21.15.90:0/511674401 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d0e1e9040 con 0x557d0e1d9520 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.467+0000 7f5884c00640 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f2610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5870007590 tx=0x7f5870000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d57544e6b645301 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.467+0000 7f586dc00640 1 -- 172.21.15.90:0/511674401 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700a9770 con 0x557d0e1d9520 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.468+0000 7f586dc00640 1 -- 172.21.15.90:0/511674401 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58700b25d0 con 0x557d0e1d9520 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.468+0000 7f586dc00640 1 -- 172.21.15.90:0/511674401 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700a8730 con 0x557d0e1d9520 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.468+0000 7f588ada5740 1 -- 172.21.15.90:0/511674401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 msgr2=0x557d0e1f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.468+0000 7f588ada5740 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f2610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5870007590 tx=0x7f5870000bc0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.470+0000 7f588ada5740 1 -- 172.21.15.90:0/511674401 shutdown_connections 2024-06-02T15:53:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.470+0000 7f588ada5740 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.470+0000 7f588ada5740 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f2610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.470+0000 7f588ada5740 1 --2- 172.21.15.90:0/511674401 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.470+0000 7f588ada5740 1 -- 172.21.15.90:0/511674401 >> 172.21.15.90:0/511674401 conn(0x557d0e1d41a0 msgr2=0x557d0e1d9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.471+0000 7f588ada5740 1 -- 172.21.15.90:0/511674401 shutdown_connections 2024-06-02T15:53:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.471+0000 7f588ada5740 1 -- 172.21.15.90:0/511674401 wait complete. 2024-06-02T15:53:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.472+0000 7f588ada5740 1 -- start start 2024-06-02T15:53:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.472+0000 7f588ada5740 1 Processor -- start 2024-06-02T15:53:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.473+0000 7f588ada5740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.473+0000 7f588ada5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.473+0000 7f588ada5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.473+0000 7f5885600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f5885600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42618/0 (socket says 172.21.15.90:42618) 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f5884c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f5884c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57156/0 (socket says 172.21.15.90:57156) 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f5885600640 1 -- 172.21.15.90:0/754574031 learned_addr learned my addr 172.21.15.90:0/754574031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f5886000640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a79d0 con 0x557d0e1de690 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a8e40 con 0x557d0e1d8b30 2024-06-02T15:53:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a80a0 con 0x557d0e1d9520 2024-06-02T15:53:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.474+0000 7f588ada5740 10 client.0 _ll_register_callbacks cb 0x7f5870007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 -- 172.21.15.90:0/754574031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 msgr2=0x557d0e1f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 -- 172.21.15.90:0/754574031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 msgr2=0x557d0e1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d0e1cf440 con 0x557d0e1d8b30 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5885600640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d28393343148c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5862a00640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874035040 con 0x557d0e1d8b30 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.475+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f58740035f0 con 0x557d0e1d8b30 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.476+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58700b25d0 con 0x557d0e1d8b30 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.476+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f58740037f0 con 0x557d0e1d8b30 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.476+0000 7f588ada5740 10 client.26300 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.476+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d0e1f26b0 con 0x557d0e1d8b30 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.476+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58700b25d0 con 0x557d0e1d8b30 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5874031c80 con 0x557d0e1d8b30 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 10 client.26300 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 20 client.26300 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e27b0a0 0x557d0e27d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 4 client.26300 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:53:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.477+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d0e1442c0 con 0x557d0e27b0a0 2024-06-02T15:53:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.478+0000 7f5884c00640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e27b0a0 0x557d0e27d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.479+0000 7f5884c00640 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e27b0a0 0x557d0e27d4c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26287609bc19a5af server_cookie=8b4c5ac3d686e316 in_seq=0 out_seq=0 2024-06-02T15:53:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.479+0000 7f5862a00640 10 client.26300 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.497+0000 7f5862a00640 1 -- 172.21.15.90:0/754574031 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557d0e1442c0 con 0x557d0e27b0a0 2024-06-02T15:53:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.497+0000 7f5862a00640 10 client.26300 handle_command_reply: tid=0 2024-06-02T15:53:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.497+0000 7f588ada5740 1 client.26300 shutdown 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 msgr2=0x557d0e1ee5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f5874004040 comp rx=0 tx=0).stop 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 shutdown_connections 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x557d0e27b0a0 0x557d0e27d4c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557d0e1de690 0x557d0e1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557d0e1d9520 0x557d0e1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 --2- 172.21.15.90:0/754574031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557d0e1d8b30 0x557d0e1ee5f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.498+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 >> 172.21.15.90:0/754574031 conn(0x557d0e1d41a0 msgr2=0x557d0e1da7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:53:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:53:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:53:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.500+0000 7f5862a00640 0 client.26300 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.500+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 shutdown_connections 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.500+0000 7f588ada5740 1 -- 172.21.15.90:0/754574031 wait complete. 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.500+0000 7f588ada5740 20 client.26300 trim_cache size 0 max 16384 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.501+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 msgr2=0x7f58680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.501+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 0x7f58680b4f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5878012cc0 tx=0x7f587801c000 comp rx=0 tx=0).stop 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.501+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 msgr2=0x7f5880101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.501+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 0x7f5880101520 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f587000c940 tx=0x7f587000ce00 comp rx=0 tx=0).stop 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.501+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 shutdown_connections 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f58680b2a70 0x7f58680b4f30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58801087d0 0x7f5880101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5880105ea0 0x7f5880101520 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 --2- 172.21.15.90:0/735612672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5880103570 0x7f5880102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 >> 172.21.15.90:0/735612672 conn(0x7f58800fcf20 msgr2=0x7f5880103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 shutdown_connections 2024-06-02T15:53:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:02.502+0000 7f5888000640 1 -- 172.21.15.90:0/735612672 wait complete. 2024-06-02T15:53:02.793 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:53:03.241 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:03 smithi090 ceph-mon[31595]: pgmap v772: 129 pgs: 129 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-02T15:53:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:03 smithi120 ceph-mon[36451]: pgmap v772: 129 pgs: 129 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-02T15:53:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:03 smithi135 ceph-mon[36839]: pgmap v772: 129 pgs: 129 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-06-02T15:53:05.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.093+0000 7f5817600700 1 -- 172.21.15.90:0/3058528725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810114770 msgr2=0x7f5810116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.093+0000 7f5817600700 1 --2- 172.21.15.90:0/3058528725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810114770 0x7f5810116b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5808009990 tx=0x7f5808009ca0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 -- 172.21.15.90:0/3058528725 shutdown_connections 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 --2- 172.21.15.90:0/3058528725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810114770 0x7f5810116b60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 --2- 172.21.15.90:0/3058528725 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810075770 0x7f5810075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 --2- 172.21.15.90:0/3058528725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810076090 0x7f5810114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 -- 172.21.15.90:0/3058528725 >> 172.21.15.90:0/3058528725 conn(0x7f581007a6a0 msgr2=0x7f581007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 -- 172.21.15.90:0/3058528725 shutdown_connections 2024-06-02T15:53:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.095+0000 7f5817600700 1 -- 172.21.15.90:0/3058528725 wait complete. 2024-06-02T15:53:05.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.097+0000 7f5817600700 1 Processor -- start 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.098+0000 7f5817600700 1 -- start start 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 0x7f58101a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 0x7f58101a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810114770 0x7f58101a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5810119df0 con 0x7f5810075770 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5810119c70 con 0x7f5810114770 2024-06-02T15:53:05.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.099+0000 7f5817600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5810119f70 con 0x7f5810076090 2024-06-02T15:53:05.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.100+0000 7f5807e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 0x7f58101a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.100+0000 7f5807e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 0x7f58101a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57188/0 (socket says 172.21.15.90:57188) 2024-06-02T15:53:05.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.100+0000 7f5807e00700 1 -- 172.21.15.90:0/4091542414 learned_addr learned my addr 172.21.15.90:0/4091542414 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:05.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.100+0000 7f5815600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810114770 0x7f58101a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.101+0000 7f5814c00700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 0x7f58101a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.103+0000 7f5814c00700 1 -- 172.21.15.90:0/4091542414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 msgr2=0x7f58101a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.103+0000 7f5814c00700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 0x7f58101a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.103+0000 7f5814c00700 1 -- 172.21.15.90:0/4091542414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810114770 msgr2=0x7f58101a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.103+0000 7f5814c00700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810114770 0x7f58101a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.103+0000 7f5814c00700 1 -- 172.21.15.90:0/4091542414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58101a5230 con 0x7f5810075770 2024-06-02T15:53:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.105+0000 7f5814c00700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 0x7f58101a0220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f580000bd70 tx=0x7f580000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.106+0000 7f580e000700 1 -- 172.21.15.90:0/4091542414 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800019070 con 0x7f5810075770 2024-06-02T15:53:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.106+0000 7f580e000700 1 -- 172.21.15.90:0/4091542414 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580000cc10 con 0x7f5810075770 2024-06-02T15:53:05.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.106+0000 7f580e000700 1 -- 172.21.15.90:0/4091542414 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000125a0 con 0x7f5810075770 2024-06-02T15:53:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.107+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58101a5520 con 0x7f5810075770 2024-06-02T15:53:05.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.107+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5810107a00 con 0x7f5810075770 2024-06-02T15:53:05.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.113+0000 7f580e000700 1 -- 172.21.15.90:0/4091542414 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5800014440 con 0x7f5810075770 2024-06-02T15:53:05.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.113+0000 7f5821fa0b80 1 Processor -- start 2024-06-02T15:53:05.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.115+0000 7f580e000700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 0x7f57f40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 -- start start 2024-06-02T15:53:05.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7966fae40 con 0x55f7967a1eb0 2024-06-02T15:53:05.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f7966ea9e0 con 0x55f7967a27d0 2024-06-02T15:53:05.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f7966eea20 con 0x55f79679c880 2024-06-02T15:53:05.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f580e000700 1 -- 172.21.15.90:0/4091542414 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f58000a5340 con 0x7f5810075770 2024-06-02T15:53:05.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.117+0000 7f5815600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 0x7f57f40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.119+0000 7f5814c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.119+0000 7f5814c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44462/0 (socket says 172.21.15.90:44462) 2024-06-02T15:53:05.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.120+0000 7f5814c00700 1 -- 172.21.15.90:0/3772980958 learned_addr learned my addr 172.21.15.90:0/3772980958 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:05.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.120+0000 7f5807e00700 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.119+0000 7f5815600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.120+0000 7f5815600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 0x7f57f40b4ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f580800b2e0 tx=0x7f5808005370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 -- 172.21.15.90:0/3772980958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 msgr2=0x55f79679c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 -- 172.21.15.90:0/3772980958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 msgr2=0x55f7967a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 -- 172.21.15.90:0/3772980958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7966b3420 con 0x55f79679c880 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5814c00700 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.121+0000 7f5815600700 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679cce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5808009750 tx=0x7f580802a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c483c8b93dd20a8a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.122+0000 7f5806000700 1 -- 172.21.15.90:0/3772980958 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808021040 con 0x55f79679c880 2024-06-02T15:53:05.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.123+0000 7f5806000700 1 -- 172.21.15.90:0/3772980958 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580802d070 con 0x55f79679c880 2024-06-02T15:53:05.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.123+0000 7f5806000700 1 -- 172.21.15.90:0/3772980958 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800fbb0 con 0x55f79679c880 2024-06-02T15:53:05.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.123+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3772980958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 msgr2=0x55f79679cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.123+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679cce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5808009750 tx=0x7f580802a000 comp rx=0 tx=0).stop 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3772980958 shutdown_connections 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679cce0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3772980958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3772980958 >> 172.21.15.90:0/3772980958 conn(0x55f7966cabe0 msgr2=0x55f796797010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3772980958 shutdown_connections 2024-06-02T15:53:05.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.124+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3772980958 wait complete. 2024-06-02T15:53:05.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.125+0000 7f5821fa0b80 1 -- start start 2024-06-02T15:53:05.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.125+0000 7f5821fa0b80 1 Processor -- start 2024-06-02T15:53:05.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7965f0fa0 con 0x55f7967a1eb0 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f79668b760 con 0x55f79679c880 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f7967a91e0 con 0x55f7967a27d0 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5821fa0b80 10 client.0 _ll_register_callbacks cb 0x55f796694c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:05.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5814c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5814c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42658/0 (socket says 172.21.15.90:42658) 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5814c00700 1 -- 172.21.15.90:0/3971259710 learned_addr learned my addr 172.21.15.90:0/3971259710 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.126+0000 7f5815600700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 -- 172.21.15.90:0/3971259710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 msgr2=0x55f7967a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 -- 172.21.15.90:0/3971259710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 msgr2=0x55f7967a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 -- 172.21.15.90:0/3971259710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79663a8d0 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.127+0000 7f5814c00700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5800004c20 tx=0x7f58000a3400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b531b636d0b3989 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f57fea00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000a5540 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5800014730 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580802d070 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f58000a56e0 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f5821fa0b80 10 client.26309 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f7966bb060 con 0x55f79679c880 2024-06-02T15:53:05.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.128+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580802d070 con 0x55f79679c880 2024-06-02T15:53:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f58000c2060 con 0x55f79679c880 2024-06-02T15:53:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 10 client.26309 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 20 client.26309 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7967b5ce0 0x55f7967b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 4 client.26309 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.129+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7966fcd90 con 0x55f7967b5ce0 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.130+0000 7f5807e00700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7967b5ce0 0x55f7967b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.130+0000 7f5807e00700 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7967b5ce0 0x55f7967b8140 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9622b2d72b75f8 server_cookie=be94a470a7202c58 in_seq=0 out_seq=0 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.130+0000 7f57fea00700 10 client.26309 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.131+0000 7f57fea00700 1 -- 172.21.15.90:0/3971259710 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f7966fcd90 con 0x55f7967b5ce0 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.131+0000 7f57fea00700 10 client.26309 handle_command_reply: tid=0 2024-06-02T15:53:05.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.131+0000 7f5821fa0b80 1 client.26309 shutdown 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 msgr2=0x55f7967a84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5800004c20 tx=0x7f58000a3400 comp rx=0 tx=0).stop 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 shutdown_connections 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f7967b5ce0 0x55f7967b8140 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967a84f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3971259710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967a5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 >> 172.21.15.90:0/3971259710 conn(0x55f7966cabe0 msgr2=0x55f79679b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.132+0000 7f57fea00700 0 client.26309 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:05.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.133+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 shutdown_connections 2024-06-02T15:53:05.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.133+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3971259710 wait complete. 2024-06-02T15:53:05.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.133+0000 7f5821fa0b80 20 client.26309 trim_cache size 0 max 16384 2024-06-02T15:53:05.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 Processor -- start 2024-06-02T15:53:05.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 -- start start 2024-06-02T15:53:05.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58101c0ac0 con 0x55f7967a1eb0 2024-06-02T15:53:05.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58101be290 con 0x55f7967a27d0 2024-06-02T15:53:05.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.158+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f581010e540 con 0x55f79679c880 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.159+0000 7f5815600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.159+0000 7f5815600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42674/0 (socket says 172.21.15.90:42674) 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.159+0000 7f5815600700 1 -- 172.21.15.90:0/3018217328 learned_addr learned my addr 172.21.15.90:0/3018217328 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.159+0000 7f5807e00700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.159+0000 7f5814c00700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5814c00700 1 -- 172.21.15.90:0/3018217328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 msgr2=0x55f79679cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5814c00700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5814c00700 1 -- 172.21.15.90:0/3018217328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 msgr2=0x55f7967b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5814c00700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5814c00700 1 -- 172.21.15.90:0/3018217328 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79678f190 con 0x55f79679c880 2024-06-02T15:53:05.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.160+0000 7f5815600700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.161+0000 7f5814c00700 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679c260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5800014a20 tx=0x7f580000bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a1e84068a9ed1c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.161+0000 7f5805600700 1 -- 172.21.15.90:0/3018217328 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000b5070 con 0x55f79679c880 2024-06-02T15:53:05.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.161+0000 7f5805600700 1 -- 172.21.15.90:0/3018217328 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58000a84c0 con 0x55f79679c880 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.161+0000 7f5805600700 1 -- 172.21.15.90:0/3018217328 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000a58b0 con 0x55f79679c880 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3018217328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 msgr2=0x55f79679c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679c260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5800014a20 tx=0x7f580000bec0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3018217328 shutdown_connections 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f7967a27d0 0x55f79679cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f79679c880 0x55f79679c260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/3018217328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.162+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3018217328 >> 172.21.15.90:0/3018217328 conn(0x55f7966cabe0 msgr2=0x55f796790e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.163+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3018217328 shutdown_connections 2024-06-02T15:53:05.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.163+0000 7f5821fa0b80 1 -- 172.21.15.90:0/3018217328 wait complete. 2024-06-02T15:53:05.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.164+0000 7f5821fa0b80 1 -- start start 2024-06-02T15:53:05.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.164+0000 7f5821fa0b80 1 Processor -- start 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f581010ebb0 con 0x55f7967a1eb0 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5810110020 con 0x55f79679c880 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f581010f280 con 0x55f7967a27d0 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5821fa0b80 10 client.0 _ll_register_callbacks cb 0x55f796694c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5814c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5814c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42686/0 (socket says 172.21.15.90:42686) 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5814c00700 1 -- 172.21.15.90:0/2719602795 learned_addr learned my addr 172.21.15.90:0/2719602795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.165+0000 7f5815600700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.166+0000 7f5814c00700 1 -- 172.21.15.90:0/2719602795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 msgr2=0x55f7967bd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.166+0000 7f5814c00700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967bd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.166+0000 7f5814c00700 1 -- 172.21.15.90:0/2719602795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 msgr2=0x55f7967bd120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:05.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.166+0000 7f5814c00700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.167+0000 7f5814c00700 1 -- 172.21.15.90:0/2719602795 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7967bdd40 con 0x55f79679c880 2024-06-02T15:53:05.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.167+0000 7f5815600700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:05.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.167+0000 7f5814c00700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f58000a5dc0 tx=0x7f58000a4d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc847f3acb81161 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:05.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.167+0000 7f57fea00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:05.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.168+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58000b5070 con 0x55f79679c880 2024-06-02T15:53:05.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.168+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58000a84c0 con 0x55f79679c880 2024-06-02T15:53:05.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.169+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5800014730 con 0x55f79679c880 2024-06-02T15:53:05.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.169+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5800088050 con 0x55f79679c880 2024-06-02T15:53:05.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.170+0000 7f5821fa0b80 10 client.26315 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:05.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.170+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f7967bdd40 con 0x55f79679c880 2024-06-02T15:53:05.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.170+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58000a84c0 con 0x55f79679c880 2024-06-02T15:53:05.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f58000a40d0 con 0x55f79679c880 2024-06-02T15:53:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 10 client.26315 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 20 client.26315 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f796831ab0 0x55f796833ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 4 client.26315 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.171+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7966f6570 con 0x55f796831ab0 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.172+0000 7f5807e00700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f796831ab0 0x55f796833ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.173+0000 7f5807e00700 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f796831ab0 0x55f796833ed0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b2472f27f95206 server_cookie=bcf923e41d651c95 in_seq=0 out_seq=0 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.173+0000 7f57fea00700 10 client.26315 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f57fea00700 1 -- 172.21.15.90:0/2719602795 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7966f6570 con 0x55f796831ab0 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f57fea00700 10 client.26315 handle_command_reply: tid=0 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 client.26315 shutdown 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 msgr2=0x55f7967aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f58000a5dc0 tx=0x7f58000a4d70 comp rx=0 tx=0).stop 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 shutdown_connections 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55f796831ab0 0x55f796833ed0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f7967a27d0 0x55f7967bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f79679c880 0x55f7967aea90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 --2- 172.21.15.90:0/2719602795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7967a1eb0 0x55f7967bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 >> 172.21.15.90:0/2719602795 conn(0x55f7966cabe0 msgr2=0x55f7967a4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f57fea00700 0 client.26315 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 shutdown_connections 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.174+0000 7f5821fa0b80 1 -- 172.21.15.90:0/2719602795 wait complete. 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.175+0000 7f5821fa0b80 20 client.26315 trim_cache size 0 max 16384 2024-06-02T15:53:05.173 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:53:05.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 msgr2=0x7f57f40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 0x7f57f40b4ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f580800b2e0 tx=0x7f5808005370 comp rx=0 tx=0).stop 2024-06-02T15:53:05.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 msgr2=0x7f58101a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:05.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 0x7f58101a0220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f580000bd70 tx=0x7f580000bda0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 shutdown_connections 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f57f40b2a10 0x7f57f40b4ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5810114770 0x7f58101a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5810075770 0x7f58101a0220 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 --2- 172.21.15.90:0/4091542414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5810076090 0x7f58101a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 >> 172.21.15.90:0/4091542414 conn(0x7f581007a6a0 msgr2=0x7f5810110210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.176+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 shutdown_connections 2024-06-02T15:53:05.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:05.177+0000 7f5817600700 1 -- 172.21.15.90:0/4091542414 wait complete. 2024-06-02T15:53:05.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:05 smithi090 ceph-mon[31595]: pgmap v773: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-06-02T15:53:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:05 smithi120 ceph-mon[36451]: pgmap v773: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-06-02T15:53:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:05 smithi135 ceph-mon[36839]: pgmap v773: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-06-02T15:53:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:07 smithi120 ceph-mon[36451]: pgmap v774: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-06-02T15:53:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:07 smithi135 ceph-mon[36839]: pgmap v774: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-06-02T15:53:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:07 smithi090 ceph-mon[31595]: pgmap v774: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 236 op/s 2024-06-02T15:53:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:09 smithi120 ceph-mon[36451]: pgmap v775: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-06-02T15:53:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:09 smithi135 ceph-mon[36839]: pgmap v775: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-06-02T15:53:09.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:09 smithi090 ceph-mon[31595]: pgmap v775: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-06-02T15:53:11.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:11 smithi090 ceph-mon[31595]: pgmap v776: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-06-02T15:53:11.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:11 smithi120 ceph-mon[36451]: pgmap v776: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-06-02T15:53:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:11 smithi135 ceph-mon[36839]: pgmap v776: 129 pgs: 129 active+clean; 274 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-06-02T15:53:13.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:13 smithi120 ceph-mon[36451]: pgmap v777: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T15:53:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:13 smithi135 ceph-mon[36839]: pgmap v777: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T15:53:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:13 smithi090 ceph-mon[31595]: pgmap v777: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T15:53:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:15 smithi120 ceph-mon[36451]: pgmap v778: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-06-02T15:53:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:15 smithi135 ceph-mon[36839]: pgmap v778: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-06-02T15:53:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:15 smithi090 ceph-mon[31595]: pgmap v778: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 192 op/s 2024-06-02T15:53:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 -- 172.21.15.90:0/4046984962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 msgr2=0x7f713410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 --2- 172.21.15.90:0/4046984962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f713410d110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f712c009ca0 tx=0x7f712c02a8e0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 -- 172.21.15.90:0/4046984962 shutdown_connections 2024-06-02T15:53:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 --2- 172.21.15.90:0/4046984962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f713410d110 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 --2- 172.21.15.90:0/4046984962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134105ec0 0x7f7134106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 --2- 172.21.15.90:0/4046984962 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134104cd0 0x7f71341050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 -- 172.21.15.90:0/4046984962 >> 172.21.15.90:0/4046984962 conn(0x7f7134100240 msgr2=0x7f7134102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.713+0000 7f713aa00640 1 -- 172.21.15.90:0/4046984962 shutdown_connections 2024-06-02T15:53:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.714+0000 7f713aa00640 1 -- 172.21.15.90:0/4046984962 wait complete. 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.715+0000 7f713aa00640 1 Processor -- start 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 -- start start 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134104cd0 0x7f713419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 0x7f713419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f71341a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71341108f0 con 0x7f7134104cd0 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7134110770 con 0x7f7134105ec0 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f713aa00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7134110a70 con 0x7f7134106880 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f7138a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f71341a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f7133400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 0x7f713419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f7138a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f71341a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46632/0 (socket says 172.21.15.90:46632) 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f7138a00640 1 -- 172.21.15.90:0/183871266 learned_addr learned my addr 172.21.15.90:0/183871266 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.716+0000 7f7133e00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134104cd0 0x7f713419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 -- 172.21.15.90:0/183871266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 msgr2=0x7f71341a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f71341a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 -- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134104cd0 msgr2=0x7f713419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134104cd0 0x7f713419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 -- 172.21.15.90:0/183871266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a2b70 con 0x7f7134105ec0 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.717+0000 7f7133400640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 0x7f713419b4e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f712000b3f0 tx=0x7f712000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.718+0000 7f7130c00640 1 -- 172.21.15.90:0/183871266 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120015020 con 0x7f7134105ec0 2024-06-02T15:53:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.718+0000 7f7130c00640 1 -- 172.21.15.90:0/183871266 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7120011ce0 con 0x7f7134105ec0 2024-06-02T15:53:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.718+0000 7f7130c00640 1 -- 172.21.15.90:0/183871266 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120015020 con 0x7f7134105ec0 2024-06-02T15:53:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.718+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a2e60 con 0x7f7134105ec0 2024-06-02T15:53:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.718+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a32f0 con 0x7f7134105ec0 2024-06-02T15:53:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.728+0000 7f7130c00640 1 -- 172.21.15.90:0/183871266 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7120018020 con 0x7f7134105ec0 2024-06-02T15:53:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.735+0000 7f7130c00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 0x7f71180b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.735+0000 7f713d841740 1 Processor -- start 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.735+0000 7f713d841740 1 -- start start 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fd520 0x555a37a02020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555a3796cb90 con 0x555a37a02690 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a3795c730 con 0x555a379fcb30 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f713d841740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a37960770 con 0x555a379fd520 2024-06-02T15:53:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.737+0000 7f7133e00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 0x7f71180b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37448/0 (socket says 172.21.15.90:37448) 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 -- 172.21.15.90:0/1157418578 learned_addr learned my addr 172.21.15.90:0/1157418578 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 -- 172.21.15.90:0/1157418578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fd520 msgr2=0x555a37a02020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fd520 0x555a37a02020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 -- 172.21.15.90:0/1157418578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 msgr2=0x555a37a04b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a04b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.738+0000 7f7133e00640 1 -- 172.21.15.90:0/1157418578 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a378f58b0 con 0x555a379fcb30 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.739+0000 7f7133e00640 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7124015770 tx=0x7f7124015c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649466b08373afef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.736+0000 7f7130c00640 1 -- 172.21.15.90:0/183871266 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71200e1940 con 0x7f7134105ec0 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.739+0000 7f7129200640 1 -- 172.21.15.90:0/1157418578 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124013860 con 0x555a379fcb30 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.739+0000 7f7129200640 1 -- 172.21.15.90:0/1157418578 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7124022030 con 0x555a379fcb30 2024-06-02T15:53:15.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.739+0000 7f7129200640 1 -- 172.21.15.90:0/1157418578 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124026c80 con 0x555a379fcb30 2024-06-02T15:53:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.742+0000 7f7133e00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 0x7f71180b50a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7124015370 tx=0x7f7124010730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 -- 172.21.15.90:0/1157418578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 msgr2=0x555a379fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7124015770 tx=0x7f7124015c30 comp rx=0 tx=0).stop 2024-06-02T15:53:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 -- 172.21.15.90:0/1157418578 shutdown_connections 2024-06-02T15:53:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fd520 0x555a37a02020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 --2- 172.21.15.90:0/1157418578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fcb30 0x555a379fcf50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.744+0000 7f713d841740 1 -- 172.21.15.90:0/1157418578 >> 172.21.15.90:0/1157418578 conn(0x555a379f81a0 msgr2=0x555a379f85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.745+0000 7f713d841740 1 -- 172.21.15.90:0/1157418578 shutdown_connections 2024-06-02T15:53:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.745+0000 7f713d841740 1 -- 172.21.15.90:0/1157418578 wait complete. 2024-06-02T15:53:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.745+0000 7f713d841740 1 -- start start 2024-06-02T15:53:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.746+0000 7f713d841740 1 Processor -- start 2024-06-02T15:53:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.747+0000 7f713d841740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.747+0000 7f713d841740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a17f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.747+0000 7f713d841740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f713d841740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555a37642790 con 0x555a37a02690 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f713d841740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555a37904bc0 con 0x555a379fd520 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f713d841740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555a37a16650 con 0x555a379fcb30 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f7133e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f713d841740 10 client.0 _ll_register_callbacks cb 0x555a378e2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f7133e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46652/0 (socket says 172.21.15.90:46652) 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f7133e00640 1 -- 172.21.15.90:0/2695027429 learned_addr learned my addr 172.21.15.90:0/2695027429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f7133400640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a17f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.748+0000 7f7138a00640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 -- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 msgr2=0x555a37a17f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a17f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 -- 172.21.15.90:0/2695027429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 msgr2=0x555a37a1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a1ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 -- 172.21.15.90:0/2695027429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a37671210 con 0x555a379fcb30 2024-06-02T15:53:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.749+0000 7f7133e00640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f712400afd0 tx=0x7f71240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459c832bb65e7b13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.750+0000 7f711d600640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:53:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.750+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124026b80 con 0x555a379fcb30 2024-06-02T15:53:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.750+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f71240059c0 con 0x555a379fcb30 2024-06-02T15:53:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.751+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7124022030 con 0x555a379fcb30 2024-06-02T15:53:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.751+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7124026560 con 0x555a379fcb30 2024-06-02T15:53:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.751+0000 7f713d841740 10 client.26254 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.751+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555a3773d620 con 0x555a379fcb30 2024-06-02T15:53:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7124022030 con 0x555a379fcb30 2024-06-02T15:53:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7124015050 con 0x555a379fcb30 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 10 client.26254 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 20 client.26254 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a231c0 0x555a37a25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 4 client.26254 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.752+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a3796eae0 con 0x555a37a231c0 2024-06-02T15:53:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.753+0000 7f7133400640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a231c0 0x555a37a25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.753+0000 7f7133400640 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a231c0 0x555a37a25610 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6389ba6b40365246 server_cookie=555db0ee9ea08e9a in_seq=0 out_seq=0 2024-06-02T15:53:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.753+0000 7f711d600640 10 client.26254 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.754+0000 7f711d600640 1 -- 172.21.15.90:0/2695027429 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555a3796eae0 con 0x555a37a231c0 2024-06-02T15:53:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.755+0000 7f711d600640 10 client.26254 handle_command_reply: tid=0 2024-06-02T15:53:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.755+0000 7f713d841740 1 client.26254 shutdown 2024-06-02T15:53:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.756+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 msgr2=0x555a37a179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.756+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f712400afd0 tx=0x7f71240046c0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.756+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 shutdown_connections 2024-06-02T15:53:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.756+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a231c0 0x555a37a25610 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.758+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a37a02690 0x555a37a1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.759+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a17f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.759+0000 7f713d841740 1 --2- 172.21.15.90:0/2695027429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a179d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.759+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 >> 172.21.15.90:0/2695027429 conn(0x555a379f81a0 msgr2=0x555a379f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.759+0000 7f711d600640 0 client.26254 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.760+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 shutdown_connections 2024-06-02T15:53:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.760+0000 7f713d841740 1 -- 172.21.15.90:0/2695027429 wait complete. 2024-06-02T15:53:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.760+0000 7f713d841740 20 client.26254 trim_cache size 0 max 16384 2024-06-02T15:53:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.792+0000 7f713d841740 1 Processor -- start 2024-06-02T15:53:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.792+0000 7f713d841740 1 -- start start 2024-06-02T15:53:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.792+0000 7f713d841740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 0x555a37a17e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f713d841740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a1ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f713d841740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 0x555a379fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f713d841740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71341b47c0 con 0x555a379fcb30 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f713d841740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f71340726c0 con 0x555a379fd520 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f713d841740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7134070510 con 0x555a37a02690 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f7138a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 0x555a379fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f7138a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 0x555a379fe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:46668/0 (socket says 172.21.15.90:46668) 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f7138a00640 1 -- 172.21.15.90:0/3221331386 learned_addr learned my addr 172.21.15.90:0/3221331386 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.793+0000 7f7133e00640 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 0x555a37a17e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.794+0000 7f7133400640 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a1ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.794+0000 7f7133e00640 1 -- 172.21.15.90:0/3221331386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 msgr2=0x555a379fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.795+0000 7f7133e00640 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 0x555a379fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.795+0000 7f7133e00640 1 -- 172.21.15.90:0/3221331386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 msgr2=0x555a37a1ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.795+0000 7f7133e00640 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a1ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.795+0000 7f7133e00640 1 -- 172.21.15.90:0/3221331386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a379f7fd0 con 0x555a379fcb30 2024-06-02T15:53:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.795+0000 7f7133e00640 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 0x555a37a17e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7124035040 tx=0x7f712400a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464a318d48cec874 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.797+0000 7f711fe00640 1 -- 172.21.15.90:0/3221331386 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400ad80 con 0x555a379fcb30 2024-06-02T15:53:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.797+0000 7f711fe00640 1 -- 172.21.15.90:0/3221331386 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7124022040 con 0x555a379fcb30 2024-06-02T15:53:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.797+0000 7f711fe00640 1 -- 172.21.15.90:0/3221331386 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240059c0 con 0x555a379fcb30 2024-06-02T15:53:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.798+0000 7f713d841740 1 -- 172.21.15.90:0/3221331386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 msgr2=0x555a37a17e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.798+0000 7f713d841740 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 0x555a37a17e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7124035040 tx=0x7f712400a450 comp rx=0 tx=0).stop 2024-06-02T15:53:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 -- 172.21.15.90:0/3221331386 shutdown_connections 2024-06-02T15:53:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a37a02690 0x555a379fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a379fd520 0x555a37a1ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 --2- 172.21.15.90:0/3221331386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fcb30 0x555a37a17e70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 -- 172.21.15.90:0/3221331386 >> 172.21.15.90:0/3221331386 conn(0x555a379f81a0 msgr2=0x555a379fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 -- 172.21.15.90:0/3221331386 shutdown_connections 2024-06-02T15:53:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.799+0000 7f713d841740 1 -- 172.21.15.90:0/3221331386 wait complete. 2024-06-02T15:53:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.800+0000 7f713d841740 1 -- start start 2024-06-02T15:53:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.800+0000 7f713d841740 1 Processor -- start 2024-06-02T15:53:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.804+0000 7f713d841740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.804+0000 7f713d841740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fd520 0x555a37a1ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.804+0000 7f713d841740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.804+0000 7f7133e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f7138a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f7133400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fd520 0x555a37a1ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f7138a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37462/0 (socket says 172.21.15.90:37462) 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f7138a00640 1 -- 172.21.15.90:0/1081997632 learned_addr learned my addr 172.21.15.90:0/1081997632 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7134070b80 con 0x555a379fd520 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7134071ff0 con 0x555a37a02690 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7134071250 con 0x555a379fcb30 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.805+0000 7f713d841740 10 client.0 _ll_register_callbacks cb 0x7f712401c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7138a00640 1 -- 172.21.15.90:0/1081997632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 msgr2=0x555a37a1c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7138a00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a1c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7138a00640 1 -- 172.21.15.90:0/1081997632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fd520 msgr2=0x555a37a1ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7138a00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fd520 0x555a37a1ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7138a00640 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a379f7a90 con 0x555a37a02690 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.806+0000 7f7133e00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.807+0000 7f7138a00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f712c031cf0 tx=0x7f712c031d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81bcbcf295410638 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.807+0000 7f711d600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.808+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c049060 con 0x555a37a02690 2024-06-02T15:53:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.808+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f712c003b80 con 0x555a37a02690 2024-06-02T15:53:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.808+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7124022040 con 0x555a37a02690 2024-06-02T15:53:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.809+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f712c031e40 con 0x555a37a02690 2024-06-02T15:53:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.809+0000 7f713d841740 10 client.26330 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.809+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555a379f8e70 con 0x555a37a02690 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.810+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7124022040 con 0x555a37a02690 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.811+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f712c004470 con 0x555a37a02690 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.811+0000 7f713d841740 10 client.26330 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.811+0000 7f713d841740 10 client.26330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.812+0000 7f713d841740 20 client.26330 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.812+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a8e310 0x555a37a90730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.812+0000 7f713d841740 4 client.26330 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:53:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.812+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a379682c0 con 0x555a37a8e310 2024-06-02T15:53:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.812+0000 7f7133e00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a8e310 0x555a37a90730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.813+0000 7f7133e00640 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a8e310 0x555a37a90730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b085985ae3975d90 server_cookie=7bb8d5c758d4a952 in_seq=0 out_seq=0 2024-06-02T15:53:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.813+0000 7f711d600640 10 client.26330 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.849+0000 7f711d600640 1 -- 172.21.15.90:0/1081997632 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555a379682c0 con 0x555a37a8e310 2024-06-02T15:53:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f711d600640 10 client.26330 handle_command_reply: tid=0 2024-06-02T15:53:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f713d841740 1 client.26330 shutdown 2024-06-02T15:53:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 msgr2=0x555a37a1d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f712c031cf0 tx=0x7f712c031d20 comp rx=0 tx=0).stop 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 shutdown_connections 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.850+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555a37a8e310 0x555a37a90730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555a37a02690 0x555a37a1d9d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555a379fd520 0x555a37a1ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 --2- 172.21.15.90:0/1081997632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555a379fcb30 0x555a37a1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 >> 172.21.15.90:0/1081997632 conn(0x555a379f81a0 msgr2=0x555a379f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f711d600640 0 client.26330 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 shutdown_connections 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.851+0000 7f713d841740 1 -- 172.21.15.90:0/1081997632 wait complete. 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.852+0000 7f713d841740 20 client.26330 trim_cache size 0 max 16384 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:53:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:53:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 msgr2=0x7f71180b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 0x7f71180b50a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7124015370 tx=0x7f7124010730 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 msgr2=0x7f713419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 0x7f713419b4e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f712000b3f0 tx=0x7f712000b8b0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 shutdown_connections 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f71180b2be0 0x7f71180b50a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7134106880 0x7f71341a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7134105ec0 0x7f713419b4e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 --2- 172.21.15.90:0/183871266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7134104cd0 0x7f713419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.853+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 >> 172.21.15.90:0/183871266 conn(0x7f7134100240 msgr2=0x7f7134109240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.854+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 shutdown_connections 2024-06-02T15:53:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:15.854+0000 7f713aa00640 1 -- 172.21.15.90:0/183871266 wait complete. 2024-06-02T15:53:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:17 smithi120 ceph-mon[36451]: pgmap v779: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2024-06-02T15:53:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:17 smithi135 ceph-mon[36839]: pgmap v779: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2024-06-02T15:53:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:17 smithi090 ceph-mon[31595]: pgmap v779: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 262 op/s 2024-06-02T15:53:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:18 smithi120 ceph-mon[36451]: pgmap v780: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 867 KiB/s wr, 204 op/s 2024-06-02T15:53:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:18 smithi135 ceph-mon[36839]: pgmap v780: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 867 KiB/s wr, 204 op/s 2024-06-02T15:53:18.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:18 smithi090 ceph-mon[31595]: pgmap v780: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 867 KiB/s wr, 204 op/s 2024-06-02T15:53:20.390 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:53:20.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:53:21.064 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:53:21.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:21 smithi120 ceph-mon[36451]: pgmap v781: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 863 KiB/s wr, 210 op/s 2024-06-02T15:53:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:21 smithi135 ceph-mon[36839]: pgmap v781: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 863 KiB/s wr, 210 op/s 2024-06-02T15:53:21.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:21 smithi090 ceph-mon[31595]: pgmap v781: 129 pgs: 129 active+clean; 284 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 863 KiB/s wr, 210 op/s 2024-06-02T15:53:23.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:23 smithi090 ceph-mon[31595]: pgmap v782: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 240 op/s 2024-06-02T15:53:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:23 smithi120 ceph-mon[36451]: pgmap v782: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 240 op/s 2024-06-02T15:53:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:23 smithi135 ceph-mon[36839]: pgmap v782: 129 pgs: 129 active+clean; 288 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 240 op/s 2024-06-02T15:53:24.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 -- 172.21.15.90:0/2266473540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 msgr2=0x7fc2c406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 --2- 172.21.15.90:0/2266473540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c406b6a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009990 tx=0x7fc2b0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 -- 172.21.15.90:0/2266473540 shutdown_connections 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 --2- 172.21.15.90:0/2266473540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 --2- 172.21.15.90:0/2266473540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 0x7fc2c4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 --2- 172.21.15.90:0/2266473540 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c406b6a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.486+0000 7fc2cc600700 1 -- 172.21.15.90:0/2266473540 >> 172.21.15.90:0/2266473540 conn(0x7fc2c40793f0 msgr2=0x7fc2c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.487+0000 7fc2cc600700 1 -- 172.21.15.90:0/2266473540 shutdown_connections 2024-06-02T15:53:24.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.488+0000 7fc2cc600700 1 -- 172.21.15.90:0/2266473540 wait complete. 2024-06-02T15:53:24.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.489+0000 7fc2cc600700 1 Processor -- start 2024-06-02T15:53:24.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.489+0000 7fc2cc600700 1 -- start start 2024-06-02T15:53:24.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.489+0000 7fc2cc600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.489+0000 7fc2cc600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50986/0 (socket says 172.21.15.90:50986) 2024-06-02T15:53:24.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2cc600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 0x7fc2c406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2c9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2c9c00700 1 -- 172.21.15.90:0/1846284087 learned_addr learned my addr 172.21.15.90:0/1846284087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2cc600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111590 con 0x7fc2c406bbe0 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111410 con 0x7fc2c406b220 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2ca600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 0x7fc2c406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.490+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111710 con 0x7fc2c41095b0 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.491+0000 7fc2c9200700 1 -- 172.21.15.90:0/1846284087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 msgr2=0x7fc2c406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.491+0000 7fc2c9200700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 0x7fc2c406e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.491+0000 7fc2c9200700 1 -- 172.21.15.90:0/1846284087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 msgr2=0x7fc2c4074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.491+0000 7fc2c9200700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.491+0000 7fc2c9200700 1 -- 172.21.15.90:0/1846284087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c406e8c0 con 0x7fc2c406bbe0 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.492+0000 7fc2c9200700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c406dd90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00098e0 tx=0x7fc2b0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.492+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.492+0000 7fc2c2000700 1 -- 172.21.15.90:0/1846284087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0026070 con 0x7fc2c406bbe0 2024-06-02T15:53:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.492+0000 7fc2c2000700 1 -- 172.21.15.90:0/1846284087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b00054e0 con 0x7fc2c406bbe0 2024-06-02T15:53:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.492+0000 7fc2c2000700 1 -- 172.21.15.90:0/1846284087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0003b50 con 0x7fc2c406bbe0 2024-06-02T15:53:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.493+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c406eb50 con 0x7fc2c406bbe0 2024-06-02T15:53:24.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.493+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c4072a60 con 0x7fc2c406bbe0 2024-06-02T15:53:24.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.498+0000 7fc2c2000700 1 -- 172.21.15.90:0/1846284087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2b0029070 con 0x7fc2c406bbe0 2024-06-02T15:53:24.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.500+0000 7fc2d6f95b80 1 Processor -- start 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.500+0000 7fc2d6f95b80 1 -- start start 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2c2000700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 0x7fc2ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbdaaec0 con 0x55b0fbe4d1c0 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbd9aa60 con 0x55b0fbe52fb0 2024-06-02T15:53:24.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbd9eaa0 con 0x55b0fbe52690 2024-06-02T15:53:24.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2c2000700 1 -- 172.21.15.90:0/1846284087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b00e2b20 con 0x7fc2c406bbe0 2024-06-02T15:53:24.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60528/0 (socket says 172.21.15.90:60528) 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.501+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2263659372 learned_addr learned my addr 172.21.15.90:0/2263659372 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9200700 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2ca600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 0x7fc2ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2263659372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52fb0 msgr2=0x55b0fbe4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2263659372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 msgr2=0x55b0fbe4d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2263659372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fbd634a0 con 0x55b0fbe52690 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.502+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8010580 tx=0x7fc2b8010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b24e5539a1c2c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.503+0000 7fc2b6a00700 1 -- 172.21.15.90:0/2263659372 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8011640 con 0x55b0fbe52690 2024-06-02T15:53:24.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.503+0000 7fc2b6a00700 1 -- 172.21.15.90:0/2263659372 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b8011c90 con 0x55b0fbe52690 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.503+0000 7fc2b6a00700 1 -- 172.21.15.90:0/2263659372 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8019b40 con 0x55b0fbe52690 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2263659372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 msgr2=0x55b0fbe52a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8010580 tx=0x7fc2b8010890 comp rx=0 tx=0).stop 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2ca600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 0x7fc2ac0b4f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc007740 tx=0x7fc2bc007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2263659372 shutdown_connections 2024-06-02T15:53:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52690 0x55b0fbe52a70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2263659372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2263659372 >> 172.21.15.90:0/2263659372 conn(0x55b0fbd7ac60 msgr2=0x55b0fbe47710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2263659372 shutdown_connections 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.504+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2263659372 wait complete. 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.505+0000 7fc2d6f95b80 1 -- start start 2024-06-02T15:53:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.505+0000 7fc2d6f95b80 1 Processor -- start 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe58ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbca1020 con 0x55b0fbe4d1c0 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbd3b7e0 con 0x55b0fbe52690 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0fbe59a60 con 0x55b0fbe52fb0 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.506+0000 7fc2d6f95b80 10 client.0 _ll_register_callbacks cb 0x55b0fbd44cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe58ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe58ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38690/0 (socket says 172.21.15.90:38690) 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2141194555 learned_addr learned my addr 172.21.15.90:0/2141194555 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2c9200700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 -- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 msgr2=0x55b0fbe56080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe56080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 -- 172.21.15.90:0/2141194555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 msgr2=0x55b0fbe58ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe58ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.507+0000 7fc2ca600700 1 -- 172.21.15.90:0/2141194555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fbcea950 con 0x55b0fbe52fb0 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.508+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe565c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55b0fbe4db80 tx=0x7fc2bc007950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=578f9e26cd730cb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.508+0000 7fc2ab400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.508+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc01b050 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.508+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b8011c90 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.508+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc2bc004b40 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.509+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc2bc02d050 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.509+0000 7fc2d6f95b80 10 client.26269 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.509+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b0fbd6b0e0 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.509+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b8011c90 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.509+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fc2bc005520 con 0x55b0fbe52fb0 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 10 client.26269 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:24.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 20 client.26269 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbe666f0 0x55b0fbe68b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 4 client.26269 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0fbdace10 con 0x55b0fbe666f0 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.510+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbe666f0 0x55b0fbe68b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.511+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbe666f0 0x55b0fbe68b30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c9c4145fabf523 server_cookie=3319e5b5062eb2 in_seq=0 out_seq=0 2024-06-02T15:53:24.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.511+0000 7fc2ab400700 10 client.26269 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.516+0000 7fc2ab400700 1 -- 172.21.15.90:0/2141194555 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b0fbdace10 con 0x55b0fbe666f0 2024-06-02T15:53:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.516+0000 7fc2ab400700 10 client.26269 handle_command_reply: tid=0 2024-06-02T15:53:24.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.516+0000 7fc2d6f95b80 1 client.26269 shutdown 2024-06-02T15:53:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 msgr2=0x55b0fbe565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe565c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55b0fbe4db80 tx=0x7fc2bc007950 comp rx=0 tx=0).stop 2024-06-02T15:53:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 shutdown_connections 2024-06-02T15:53:24.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbe666f0 0x55b0fbe68b30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe565c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe58ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2141194555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe56080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 >> 172.21.15.90:0/2141194555 conn(0x55b0fbd7ac60 msgr2=0x55b0fbe46df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2ab400700 0 client.26269 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 shutdown_connections 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2141194555 wait complete. 2024-06-02T15:53:24.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.517+0000 7fc2d6f95b80 20 client.26269 trim_cache size 0 max 16384 2024-06-02T15:53:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.551+0000 7fc2d6f95b80 1 Processor -- start 2024-06-02T15:53:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.551+0000 7fc2d6f95b80 1 -- start start 2024-06-02T15:53:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe52690 0x55b0fbe54fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe53c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41c0940 con 0x55b0fbe52690 2024-06-02T15:53:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41be110 con 0x55b0fbe4d1c0 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41bbf60 con 0x55b0fbe52fb0 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2c9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe52690 0x55b0fbe54fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2c9c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51024/0 (socket says 172.21.15.90:51024) 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2c9c00700 1 -- 172.21.15.90:0/2794664896 learned_addr learned my addr 172.21.15.90:0/2794664896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.552+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe53c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.553+0000 7fc2ca600700 1 -- 172.21.15.90:0/2794664896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 msgr2=0x55b0fbe4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.553+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.553+0000 7fc2ca600700 1 -- 172.21.15.90:0/2794664896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe52690 msgr2=0x55b0fbe54fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.553+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe52690 0x55b0fbe54fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.553+0000 7fc2ca600700 1 -- 172.21.15.90:0/2794664896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fbe545f0 con 0x55b0fbe52fb0 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.554+0000 7fc2ca600700 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe53c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc011c40 tx=0x7fc2bc0111f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92c89b3b159631b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.554+0000 7fc2b6000700 1 -- 172.21.15.90:0/2794664896 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035020 con 0x55b0fbe52fb0 2024-06-02T15:53:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.554+0000 7fc2b6000700 1 -- 172.21.15.90:0/2794664896 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc01c040 con 0x55b0fbe52fb0 2024-06-02T15:53:24.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.554+0000 7fc2b6000700 1 -- 172.21.15.90:0/2794664896 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc01b070 con 0x55b0fbe52fb0 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2794664896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 msgr2=0x55b0fbe53c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe53c20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc011c40 tx=0x7fc2bc0111f0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2794664896 shutdown_connections 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe53c20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/2794664896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe52690 0x55b0fbe54fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.555+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2794664896 >> 172.21.15.90:0/2794664896 conn(0x55b0fbd7ac60 msgr2=0x55b0fbe511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.556+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2794664896 shutdown_connections 2024-06-02T15:53:24.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.556+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/2794664896 wait complete. 2024-06-02T15:53:24.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.556+0000 7fc2d6f95b80 1 -- start start 2024-06-02T15:53:24.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 Processor -- start 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41bc5d0 con 0x55b0fbe4d1c0 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41bda40 con 0x55b0fbe52690 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41bcca0 con 0x55b0fbe52fb0 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2d6f95b80 10 client.0 _ll_register_callbacks cb 0x55b0fbd44cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:24.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.557+0000 7fc2c9200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51038/0 (socket says 172.21.15.90:51038) 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 -- 172.21.15.90:0/267117869 learned_addr learned my addr 172.21.15.90:0/267117869 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 -- 172.21.15.90:0/267117869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 msgr2=0x55b0fbe5bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe5bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 -- 172.21.15.90:0/267117869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 msgr2=0x55b0fbe5ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe5ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 -- 172.21.15.90:0/267117869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fbe5c5e0 con 0x55b0fbe52690 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2c9200700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00e0210 tx=0x7fc2b0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c4f830a9f43efb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2ab400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:24.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00a9dc0 con 0x55b0fbe52690 2024-06-02T15:53:24.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc01c040 con 0x55b0fbe52690 2024-06-02T15:53:24.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.558+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fc2b00a9230 con 0x55b0fbe52690 2024-06-02T15:53:24.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.559+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc2b00ae170 con 0x55b0fbe52690 2024-06-02T15:53:24.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.559+0000 7fc2d6f95b80 10 client.26348 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:24.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.559+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b0fbee2fd0 con 0x55b0fbe52690 2024-06-02T15:53:24.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.559+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc01c040 con 0x55b0fbe52690 2024-06-02T15:53:24.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fc2b00a8a90 con 0x55b0fbe52690 2024-06-02T15:53:24.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 10 client.26348 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:24.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:24.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 20 client.26348 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbee36f0 0x55b0fbee5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 4 client.26348 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.560+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b0fbda65f0 con 0x55b0fbee36f0 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.561+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbee36f0 0x55b0fbee5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.561+0000 7fc2c9c00700 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbee36f0 0x55b0fbee5b10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c459dbb0c0b5a8 server_cookie=44818b6217c8ca7 in_seq=0 out_seq=0 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.561+0000 7fc2ab400700 10 client.26348 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.562+0000 7fc2ab400700 1 -- 172.21.15.90:0/267117869 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b0fbda65f0 con 0x55b0fbee36f0 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.562+0000 7fc2ab400700 10 client.26348 handle_command_reply: tid=0 2024-06-02T15:53:24.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.563+0000 7fc2d6f95b80 1 client.26348 shutdown 2024-06-02T15:53:24.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.563+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 msgr2=0x55b0fbe5f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.563+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00e0210 tx=0x7fc2b0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 shutdown_connections 2024-06-02T15:53:24.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55b0fbee36f0 0x55b0fbee5b10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0fbe52fb0 0x55b0fbe5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b0fbe4d1c0 0x55b0fbe5ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 --2- 172.21.15.90:0/267117869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b0fbe52690 0x55b0fbe5f470 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 >> 172.21.15.90:0/267117869 conn(0x55b0fbd7ac60 msgr2=0x55b0fbe53aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2ab400700 0 client.26348 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 shutdown_connections 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.564+0000 7fc2d6f95b80 1 -- 172.21.15.90:0/267117869 wait complete. 2024-06-02T15:53:24.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.565+0000 7fc2d6f95b80 20 client.26348 trim_cache size 0 max 16384 2024-06-02T15:53:24.563 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:53:24.563 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:53:24.563 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "e47501fb-b9a2-437a-bfa7-181d08fe61df", 2024-06-02T15:53:24.563 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:53:24.563 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 msgr2=0x7fc2ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 0x7fc2ac0b4f00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc007740 tx=0x7fc2bc007690 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 msgr2=0x7fc2c406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c406dd90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2b00098e0 tx=0x7fc2b0009f70 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 shutdown_connections 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc2ac0b2a40 0x7fc2ac0b4f00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2c41095b0 0x7fc2c406e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c406b220 0x7fc2c4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 --2- 172.21.15.90:0/1846284087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c406bbe0 0x7fc2c406dd90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 >> 172.21.15.90:0/1846284087 conn(0x7fc2c40793f0 msgr2=0x7fc2c4101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:24.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 shutdown_connections 2024-06-02T15:53:24.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:24.566+0000 7fc2cc600700 1 -- 172.21.15.90:0/1846284087 wait complete. 2024-06-02T15:53:25.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:25 smithi090 ceph-mon[31595]: pgmap v783: 129 pgs: 129 active+clean; 291 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 237 op/s 2024-06-02T15:53:25.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:53:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:25 smithi120 ceph-mon[36451]: pgmap v783: 129 pgs: 129 active+clean; 291 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 237 op/s 2024-06-02T15:53:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:25 smithi135 ceph-mon[36839]: pgmap v783: 129 pgs: 129 active+clean; 291 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 237 op/s 2024-06-02T15:53:25.966 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:53:26.481 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:53:26.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:26.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:26.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:26.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:26.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:26 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:26.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:26 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:26 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:27.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:27 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:27.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:27 smithi090 ceph-mon[31595]: pgmap v784: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-02T15:53:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:27 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:27.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:27 smithi120 ceph-mon[36451]: pgmap v784: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-02T15:53:27.565 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:27 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:27 smithi135 ceph-mon[36839]: pgmap v784: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2024-06-02T15:53:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.739+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3557737893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1041a0 msgr2=0x7fa7bc1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.739+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3557737893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc1081c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009920 tx=0x7fa7a8009c30 comp rx=0 tx=0).stop 2024-06-02T15:53:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3557737893 shutdown_connections 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3557737893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1088f0 0x7fa7bc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3557737893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3557737893 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc1081c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3557737893 >> 172.21.15.90:0/3557737893 conn(0x7fa7bc07a6a0 msgr2=0x7fa7bc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3557737893 shutdown_connections 2024-06-02T15:53:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.740+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3557737893 wait complete. 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.741+0000 7fa7c3e00700 1 Processor -- start 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.741+0000 7fa7c3e00700 1 -- start start 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc119010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1088f0 0x7fa7bc112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1115e0 con 0x7fa7bc1037f0 2024-06-02T15:53:28.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc111460 con 0x7fa7bc1088f0 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c3e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc111760 con 0x7fa7bc1041a0 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc118ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc118ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38726/0 (socket says 172.21.15.90:38726) 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c1400700 1 -- 172.21.15.90:0/3514370729 learned_addr learned my addr 172.21.15.90:0/3514370729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1088f0 0x7fa7bc112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.742+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc119010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.743+0000 7fa7c0a00700 1 -- 172.21.15.90:0/3514370729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1088f0 msgr2=0x7fa7bc112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.743+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1088f0 0x7fa7bc112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.743+0000 7fa7c0a00700 1 -- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 msgr2=0x7fa7bc118ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.743+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc118ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.743+0000 7fa7c0a00700 1 -- 172.21.15.90:0/3514370729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc1134f0 con 0x7fa7bc1041a0 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc119010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7ba000700 1 -- 172.21.15.90:0/3514370729 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8005040 con 0x7fa7bc1041a0 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7ba000700 1 -- 172.21.15.90:0/3514370729 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a8005690 con 0x7fa7bc1041a0 2024-06-02T15:53:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7ba000700 1 -- 172.21.15.90:0/3514370729 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80038e0 con 0x7fa7bc1041a0 2024-06-02T15:53:28.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc113780 con 0x7fa7bc1041a0 2024-06-02T15:53:28.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.744+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1b56c0 con 0x7fa7bc1041a0 2024-06-02T15:53:28.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.751+0000 7fa7ba000700 1 -- 172.21.15.90:0/3514370729 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa7a80051e0 con 0x7fa7bc1041a0 2024-06-02T15:53:28.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.751+0000 7fa7ce744b80 1 Processor -- start 2024-06-02T15:53:28.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.752+0000 7fa7ce744b80 1 -- start start 2024-06-02T15:53:28.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.753+0000 7fa7ba000700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 0x7fa7a40b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.754+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 0x7fa7a40b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ba000700 1 -- 172.21.15.90:0/3514370729 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7a80e3250 con 0x7fa7bc1041a0 2024-06-02T15:53:28.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311eaf30 0x5596311e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311e5140 0x5596311e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51060/0 (socket says 172.21.15.90:51060) 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559631142e40 con 0x5596311e5140 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7c1400700 1 -- 172.21.15.90:0/4293565216 learned_addr learned my addr 172.21.15.90:0/4293565216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5596311329e0 con 0x5596311ea610 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559631136a20 con 0x5596311eaf30 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.756+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 0x7fa7a40b4e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400f540 tx=0x7fa7b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311e5140 0x5596311e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311eaf30 0x5596311e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 -- 172.21.15.90:0/4293565216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311eaf30 msgr2=0x5596311e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311eaf30 0x5596311e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 -- 172.21.15.90:0/4293565216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311e5140 msgr2=0x5596311e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311e5140 0x5596311e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 -- 172.21.15.90:0/4293565216 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596310fb420 con 0x5596311ea610 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7c1400700 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b919d4e0b9697719 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.757+0000 7fa7aea00700 1 -- 172.21.15.90:0/4293565216 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004a30 con 0x5596311ea610 2024-06-02T15:53:28.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.758+0000 7fa7aea00700 1 -- 172.21.15.90:0/4293565216 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b00054d0 con 0x5596311ea610 2024-06-02T15:53:28.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.758+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 msgr2=0x5596311ea9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.758+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b0011720 comp rx=0 tx=0).stop 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 shutdown_connections 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311e5140 0x5596311e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311ea9f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/4293565216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311eaf30 0x5596311e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 >> 172.21.15.90:0/4293565216 conn(0x559631112be0 msgr2=0x5596311df690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 shutdown_connections 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.759+0000 7fa7ce744b80 1 -- 172.21.15.90:0/4293565216 wait complete. 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.760+0000 7fa7ce744b80 1 -- start start 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.760+0000 7fa7ce744b80 1 Processor -- start 2024-06-02T15:53:28.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.760+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.760+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311ea610 0x5596311f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559631038fa0 con 0x5596311eaf30 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5596310d3760 con 0x5596311e5140 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596311f1a80 con 0x5596311ea610 2024-06-02T15:53:28.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7ce744b80 10 client.0 _ll_register_callbacks cb 0x5596310dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7c1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51070/0 (socket says 172.21.15.90:51070) 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7c1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7c1400700 1 -- 172.21.15.90:0/288302829 learned_addr learned my addr 172.21.15.90:0/288302829 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.761+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311ea610 0x5596311f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 -- 172.21.15.90:0/288302829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311ea610 msgr2=0x5596311f0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311ea610 0x5596311f0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 -- 172.21.15.90:0/288302829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 msgr2=0x5596311edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 -- 172.21.15.90:0/288302829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596310828d0 con 0x5596311e5140 2024-06-02T15:53:28.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7c1400700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0019510 tx=0x7fa7b0004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9ed9e7af6e2b14 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.762+0000 7fa7a3400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0027070 con 0x5596311e5140 2024-06-02T15:53:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7b0003890 con 0x5596311e5140 2024-06-02T15:53:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b00054d0 con 0x5596311e5140 2024-06-02T15:53:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa7b0031050 con 0x5596311e5140 2024-06-02T15:53:28.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7ce744b80 10 client.26360 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559631103060 con 0x5596311e5140 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.764+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b00054d0 con 0x5596311e5140 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa7b0023a00 con 0x5596311e5140 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 10 client.26360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 20 client.26360 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:28.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5596311fe470 0x559631200860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 4 client.26360 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559631144d90 con 0x5596311fe470 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.765+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5596311fe470 0x559631200860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.766+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5596311fe470 0x559631200860 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39512a751a31e99 server_cookie=ef69480f94528ab1 in_seq=0 out_seq=0 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.766+0000 7fa7a3400700 10 client.26360 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.766+0000 7fa7a3400700 1 -- 172.21.15.90:0/288302829 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559631144d90 con 0x5596311fe470 2024-06-02T15:53:28.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.766+0000 7fa7a3400700 10 client.26360 handle_command_reply: tid=0 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.766+0000 7fa7ce744b80 1 client.26360 shutdown 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 msgr2=0x5596311f0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0019510 tx=0x7fa7b0004f10 comp rx=0 tx=0).stop 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 shutdown_connections 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5596311fe470 0x559631200860 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311e5140 0x5596311f0a80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/288302829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311ea610 0x5596311f0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7a3400700 0 client.26360 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 >> 172.21.15.90:0/288302829 conn(0x559631112be0 msgr2=0x5596311ded70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 shutdown_connections 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 1 -- 172.21.15.90:0/288302829 wait complete. 2024-06-02T15:53:28.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.767+0000 7fa7ce744b80 20 client.26360 trim_cache size 0 max 16384 2024-06-02T15:53:28.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.799+0000 7fa7ce744b80 1 Processor -- start 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.799+0000 7fa7ce744b80 1 -- start start 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c0115e0 con 0x5596311eaf30 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00edb0 con 0x5596311ea610 2024-06-02T15:53:28.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.800+0000 7fa7ce744b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00cc00 con 0x5596311e5140 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c0a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c0a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51076/0 (socket says 172.21.15.90:51076) 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c0a00700 1 -- 172.21.15.90:0/3189171978 learned_addr learned my addr 172.21.15.90:0/3189171978 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1400700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 -- 172.21.15.90:0/3189171978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 msgr2=0x5596311e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 -- 172.21.15.90:0/3189171978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 msgr2=0x5596311eca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c1e00700 1 -- 172.21.15.90:0/3189171978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596311e04a0 con 0x5596311eaf30 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.801+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:28.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.802+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311df4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5596311e5b00 tx=0x7fa7b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c5038911c3e09f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.803+0000 7fa7ae000700 1 -- 172.21.15.90:0/3189171978 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b402c670 con 0x5596311eaf30 2024-06-02T15:53:28.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.803+0000 7fa7ae000700 1 -- 172.21.15.90:0/3189171978 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b4027070 con 0x5596311eaf30 2024-06-02T15:53:28.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.803+0000 7fa7ae000700 1 -- 172.21.15.90:0/3189171978 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4015670 con 0x5596311eaf30 2024-06-02T15:53:28.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.804+0000 7fa7ce744b80 1 -- 172.21.15.90:0/3189171978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 msgr2=0x5596311df4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.804+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311df4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5596311e5b00 tx=0x7fa7b4014000 comp rx=0 tx=0).stop 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 -- 172.21.15.90:0/3189171978 shutdown_connections 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311df4f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/3189171978 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311eca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 -- 172.21.15.90:0/3189171978 >> 172.21.15.90:0/3189171978 conn(0x559631112be0 msgr2=0x5596311f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 -- 172.21.15.90:0/3189171978 shutdown_connections 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.805+0000 7fa7ce744b80 1 -- 172.21.15.90:0/3189171978 wait complete. 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.806+0000 7fa7ce744b80 1 -- start start 2024-06-02T15:53:28.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.806+0000 7fa7ce744b80 1 Processor -- start 2024-06-02T15:53:28.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.807+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.807+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.807+0000 7fa7ce744b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7c1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7c1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38772/0 (socket says 172.21.15.90:38772) 2024-06-02T15:53:28.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7c1e00700 1 -- 172.21.15.90:0/2198359556 learned_addr learned my addr 172.21.15.90:0/2198359556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:28.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00d270 con 0x5596311eaf30 2024-06-02T15:53:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00e6e0 con 0x5596311ea610 2024-06-02T15:53:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c00d940 con 0x5596311e5140 2024-06-02T15:53:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.808+0000 7fa7c1400700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7ce744b80 10 client.0 _ll_register_callbacks cb 0x5596310dcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c1e00700 1 -- 172.21.15.90:0/2198359556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 msgr2=0x5596311f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c1e00700 1 -- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 msgr2=0x5596311f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.809+0000 7fa7c1e00700 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596311f4440 con 0x5596311eaf30 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.810+0000 7fa7c0a00700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.810+0000 7fa7c1400700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.810+0000 7fa7c1e00700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e90 tx=0x7fa7b4017670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc5e98f95bcb378 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.810+0000 7fa7a3400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:53:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.810+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40090f0 con 0x5596311eaf30 2024-06-02T15:53:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.811+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4027070 con 0x5596311eaf30 2024-06-02T15:53:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.811+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa7b402c470 con 0x5596311eaf30 2024-06-02T15:53:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.811+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa7b4017e10 con 0x5596311eaf30 2024-06-02T15:53:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.811+0000 7fa7ce744b80 10 client.16068 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.811+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559631269790 con 0x5596311eaf30 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7b401b030 con 0x5596311eaf30 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa7b4033020 con 0x5596311eaf30 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 20 client.16068 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559631269e30 0x55963126c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 4 client.16068 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.812+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa7b4027070 con 0x559631269e30 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.813+0000 7fa7c1400700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559631269e30 0x55963126c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.813+0000 7fa7a3400700 10 client.16068 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.813+0000 7fa7c1400700 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559631269e30 0x55963126c250 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf01392576b57d6 server_cookie=c895b7a818456ccd in_seq=0 out_seq=0 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7a3400700 1 -- 172.21.15.90:0/2198359556 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fa7b4027070 con 0x559631269e30 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7a3400700 10 client.16068 handle_command_reply: tid=0 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 client.16068 shutdown 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 msgr2=0x5596311f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009e90 tx=0x7fa7b4017670 comp rx=0 tx=0).stop 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 shutdown_connections 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559631269e30 0x55963126c250 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596311eaf30 0x5596311f3dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596311e5140 0x5596311f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 --2- 172.21.15.90:0/2198359556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5596311ea610 0x5596311f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.814+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 >> 172.21.15.90:0/2198359556 conn(0x559631112be0 msgr2=0x5596311e9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.815+0000 7fa7a3400700 0 client.16068 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.815+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 shutdown_connections 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.815+0000 7fa7ce744b80 1 -- 172.21.15.90:0/2198359556 wait complete. 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.815+0000 7fa7ce744b80 20 client.16068 trim_cache size 0 max 16384 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-06-02T15:53:28.814 INFO:teuthology.orchestra.run.smithi090.stdout: "e47501fb-b9a2-437a-bfa7-181d08fe61df": { 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "e47501fb-b9a2-437a-bfa7-181d08fe61df", 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:53:28.815 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.816+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 msgr2=0x7fa7a40b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.816+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 0x7fa7a40b4e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400f540 tx=0x7fa7b400b040 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.816+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 msgr2=0x7fa7bc119010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.816+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc119010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a800fa70 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 shutdown_connections 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa7a40b2940 0x7fa7a40b4e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa7bc1088f0 0x7fa7bc112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7bc1037f0 0x7fa7bc118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 --2- 172.21.15.90:0/3514370729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7bc1041a0 0x7fa7bc119010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 >> 172.21.15.90:0/3514370729 conn(0x7fa7bc07a6a0 msgr2=0x7fa7bc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 shutdown_connections 2024-06-02T15:53:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:53:28.817+0000 7fa7c3e00700 1 -- 172.21.15.90:0/3514370729 wait complete. 2024-06-02T15:53:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 -- 172.21.15.90:0/2949470638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410a2d0 msgr2=0x7f5f1410a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 --2- 172.21.15.90:0/2949470638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410a2d0 0x7f5f1410a750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009900 tx=0x7f5f0002a410 comp rx=0 tx=0).stop 2024-06-02T15:53:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 -- 172.21.15.90:0/2949470638 shutdown_connections 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 --2- 172.21.15.90:0/2949470638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1410ac90 0x7f5f14111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 --2- 172.21.15.90:0/2949470638 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410a2d0 0x7f5f1410a750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 --2- 172.21.15.90:0/2949470638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f141090e0 0x7f5f14109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.090+0000 7f5f1a600640 1 -- 172.21.15.90:0/2949470638 >> 172.21.15.90:0/2949470638 conn(0x7f5f140780a0 msgr2=0x7f5f1407a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.091+0000 7f5f1a600640 1 -- 172.21.15.90:0/2949470638 shutdown_connections 2024-06-02T15:53:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.091+0000 7f5f1a600640 1 -- 172.21.15.90:0/2949470638 wait complete. 2024-06-02T15:53:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.093+0000 7f5f1a600640 1 Processor -- start 2024-06-02T15:53:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.101+0000 7f5f1a600640 1 -- start start 2024-06-02T15:53:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f141090e0 0x7f5f14080230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410ac90 0x7f5f141af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14114c70 con 0x7f5f141090e0 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14114af0 con 0x7f5f1410ac90 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f1a600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14114df0 con 0x7f5f1410a2d0 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f12a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f12a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60632/0 (socket says 172.21.15.90:60632) 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.102+0000 7f5f12a00640 1 -- 172.21.15.90:0/948206290 learned_addr learned my addr 172.21.15.90:0/948206290 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f13e00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410ac90 0x7f5f141af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f13400640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f141090e0 0x7f5f14080230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f12a00640 1 -- 172.21.15.90:0/948206290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410ac90 msgr2=0x7f5f141af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f12a00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410ac90 0x7f5f141af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f12a00640 1 -- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f141090e0 msgr2=0x7f5f14080230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f12a00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f141090e0 0x7f5f14080230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.103+0000 7f5f12a00640 1 -- 172.21.15.90:0/948206290 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1406c470 con 0x7f5f1410a2d0 2024-06-02T15:53:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.104+0000 7f5f12a00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f00031500 tx=0x7f5f00003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.104+0000 7f5f0be00640 1 -- 172.21.15.90:0/948206290 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0003f070 con 0x7f5f1410a2d0 2024-06-02T15:53:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.104+0000 7f5f0be00640 1 -- 172.21.15.90:0/948206290 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f00031c50 con 0x7f5f1410a2d0 2024-06-02T15:53:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.104+0000 7f5f0be00640 1 -- 172.21.15.90:0/948206290 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00035030 con 0x7f5f1410a2d0 2024-06-02T15:53:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.105+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1406c700 con 0x7f5f1410a2d0 2024-06-02T15:53:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.105+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1406cba0 con 0x7f5f1410a2d0 2024-06-02T15:53:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 Processor -- start 2024-06-02T15:53:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 -- start start 2024-06-02T15:53:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553db2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556553d22b90 con 0x556553db8690 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556553d12730 con 0x556553db3520 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.111+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556553d16770 con 0x556553db2b30 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51118/0 (socket says 172.21.15.90:51118) 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 -- 172.21.15.90:0/1942796230 learned_addr learned my addr 172.21.15.90:0/1942796230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13e00640 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553db2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 -- 172.21.15.90:0/1942796230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 msgr2=0x556553db2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553db2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 -- 172.21.15.90:0/1942796230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 msgr2=0x556553dbab20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.112+0000 7f5f13400640 1 -- 172.21.15.90:0/1942796230 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556553cab8b0 con 0x556553db3520 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.113+0000 7f5f0be00640 1 -- 172.21.15.90:0/948206290 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f00042070 con 0x7f5f1410a2d0 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.113+0000 7f5f13400640 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009ca0 tx=0x7f5f04011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59886cd7b12360e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.113+0000 7f5f08c00640 1 -- 172.21.15.90:0/1942796230 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04004220 con 0x556553db3520 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.113+0000 7f5f08c00640 1 -- 172.21.15.90:0/1942796230 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f040043c0 con 0x556553db3520 2024-06-02T15:53:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.113+0000 7f5f08c00640 1 -- 172.21.15.90:0/1942796230 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04019830 con 0x556553db3520 2024-06-02T15:53:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.114+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1942796230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 msgr2=0x556553db8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.114+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009ca0 tx=0x7f5f04011810 comp rx=0 tx=0).stop 2024-06-02T15:53:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.114+0000 7f5f0be00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 0x7f5ef80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1942796230 shutdown_connections 2024-06-02T15:53:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553db8020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1942796230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553db2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1942796230 >> 172.21.15.90:0/1942796230 conn(0x556553dae1a0 msgr2=0x556553dae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f0be00640 1 -- 172.21.15.90:0/948206290 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5f00102c90 con 0x7f5f1410a2d0 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1942796230 shutdown_connections 2024-06-02T15:53:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.115+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1942796230 wait complete. 2024-06-02T15:53:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.116+0000 7f5f13e00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 0x7f5ef80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.116+0000 7f5f1d3c4740 1 -- start start 2024-06-02T15:53:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.116+0000 7f5f1d3c4740 1 Processor -- start 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f13e00640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 0x7f5ef80b4eb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x556553db3d80 tx=0x7f5f0c0085f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 0x556553dbcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db8690 0x556553dc3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565539f8790 con 0x556553db3520 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556553cbabc0 con 0x556553db2b30 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556553dbb640 con 0x556553db8690 2024-06-02T15:53:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f1d3c4740 10 client.0 _ll_register_callbacks cb 0x556553c98400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.117+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51134/0 (socket says 172.21.15.90:51134) 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f12a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 0x556553dbcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 -- 172.21.15.90:0/1110189316 learned_addr learned my addr 172.21.15.90:0/1110189316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13e00640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db8690 0x556553dc3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 -- 172.21.15.90:0/1110189316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db8690 msgr2=0x556553dc3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db8690 0x556553dc3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 -- 172.21.15.90:0/1110189316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 msgr2=0x556553dbcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 0x556553dbcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.118+0000 7f5f13400640 1 -- 172.21.15.90:0/1110189316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556553a27210 con 0x556553db2b30 2024-06-02T15:53:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.119+0000 7f5f12a00640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 0x556553dbcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.119+0000 7f5f13400640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009c70 tx=0x7f5f040045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fd51868ff4d26b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.119+0000 7f5efd600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.119+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04028070 con 0x556553db2b30 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.119+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5f04005490 con 0x556553db2b30 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.120+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f040043c0 con 0x556553db2b30 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.121+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5f04032050 con 0x556553db2b30 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 10 client.26375 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:29.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556553af3620 con 0x556553db2b30 2024-06-02T15:53:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f040043c0 con 0x556553db2b30 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5f04004aa0 con 0x556553db2b30 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 10 client.26375 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 10 client.26375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 20 client.26375 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.122+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553dc8340 0x556553dca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.123+0000 7f5f1d3c4740 4 client.26375 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:29.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.123+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556553d24ae0 con 0x556553dc8340 2024-06-02T15:53:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.123+0000 7f5f12a00640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553dc8340 0x556553dca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.123+0000 7f5f12a00640 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553dc8340 0x556553dca7c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e254bc3ac8d412c server_cookie=694e634d8d7b112a in_seq=0 out_seq=0 2024-06-02T15:53:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.123+0000 7f5efd600640 10 client.26375 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.124+0000 7f5efd600640 1 -- 172.21.15.90:0/1110189316 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556553d24ae0 con 0x556553dc8340 2024-06-02T15:53:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.124+0000 7f5efd600640 10 client.26375 handle_command_reply: tid=0 2024-06-02T15:53:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.124+0000 7f5f1d3c4740 1 client.26375 shutdown 2024-06-02T15:53:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.125+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 msgr2=0x556553dbc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.125+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009c70 tx=0x7f5f040045f0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 shutdown_connections 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553dc8340 0x556553dca7c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db8690 0x556553dc3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db3520 0x556553dbcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5efd600640 0 client.26375 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1110189316 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dbc9b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 >> 172.21.15.90:0/1110189316 conn(0x556553dae1a0 msgr2=0x556553dad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.126+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 shutdown_connections 2024-06-02T15:53:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.127+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1110189316 wait complete. 2024-06-02T15:53:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.127+0000 7f5f1d3c4740 20 client.26375 trim_cache size 0 max 16384 2024-06-02T15:53:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.150+0000 7f5f1d3c4740 1 Processor -- start 2024-06-02T15:53:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.150+0000 7f5f1d3c4740 1 -- start start 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553dc3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef40125a0 con 0x556553db8690 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef400fdb0 con 0x556553db3520 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef400dc00 con 0x556553db2b30 2024-06-02T15:53:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60668/0 (socket says 172.21.15.90:60668) 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 -- 172.21.15.90:0/1078047571 learned_addr learned my addr 172.21.15.90:0/1078047571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f12a00640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553dc3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 -- 172.21.15.90:0/1078047571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 msgr2=0x556553dc3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553dc3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13e00640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.151+0000 7f5f13400640 1 -- 172.21.15.90:0/1078047571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 msgr2=0x556553dbbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5f13400640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5f13400640 1 -- 172.21.15.90:0/1078047571 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556553db32c0 con 0x556553db2b30 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5f13400640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f04011850 tx=0x7f5f04005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c37fa94ad6ac7cc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5f12a00640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553dc3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5f13e00640 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5effe00640 1 -- 172.21.15.90:0/1078047571 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0403c070 con 0x556553db2b30 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5effe00640 1 -- 172.21.15.90:0/1078047571 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f04004b10 con 0x556553db2b30 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.152+0000 7f5effe00640 1 -- 172.21.15.90:0/1078047571 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04028070 con 0x556553db2b30 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1078047571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 msgr2=0x556553dbba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f04011850 tx=0x7f5f04005970 comp rx=0 tx=0).stop 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1078047571 shutdown_connections 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dbbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db3520 0x556553dc3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/1078047571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db2b30 0x556553dbba70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.153+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1078047571 >> 172.21.15.90:0/1078047571 conn(0x556553dae1a0 msgr2=0x556553db1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.154+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1078047571 shutdown_connections 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.154+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/1078047571 wait complete. 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.154+0000 7f5f1d3c4740 1 -- start start 2024-06-02T15:53:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.154+0000 7f5f1d3c4740 1 Processor -- start 2024-06-02T15:53:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 0x556553dc07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dc0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef400e270 con 0x556553db8690 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef400f6e0 con 0x556553db2b30 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef400e940 con 0x556553db3520 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f1d3c4740 10 client.0 _ll_register_callbacks cb 0x7f5f0c008490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f12a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 0x556553dc07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51158/0 (socket says 172.21.15.90:51158) 2024-06-02T15:53:29.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 -- 172.21.15.90:0/448954644 learned_addr learned my addr 172.21.15.90:0/448954644 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13e00640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dc0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 -- 172.21.15.90:0/448954644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 msgr2=0x556553dc07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 0x556553dc07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 -- 172.21.15.90:0/448954644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 msgr2=0x556553dc0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dc0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f13400640 1 -- 172.21.15.90:0/448954644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556553dc1390 con 0x556553db2b30 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.155+0000 7f5f12a00640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 0x556553dc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.156+0000 7f5f13400640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009c70 tx=0x7f5f040056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130772124f89569 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.156+0000 7f5efd600640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.156+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0403c030 con 0x556553db2b30 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.156+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f5f04019a00 con 0x556553db2b30 2024-06-02T15:53:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.156+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f04004b10 con 0x556553db2b30 2024-06-02T15:53:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.157+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5f04028070 con 0x556553db2b30 2024-06-02T15:53:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.157+0000 7f5f1d3c4740 10 client.26378 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.157+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556553e32000 con 0x556553db2b30 2024-06-02T15:53:29.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.157+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f04004b10 con 0x556553db2b30 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5f04019c00 con 0x556553db2b30 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 10 client.26378 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 20 client.26378 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553e328f0 0x556553e34d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 4 client.26378 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556553d1e2c0 con 0x556553e328f0 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.158+0000 7f5f12a00640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553e328f0 0x556553e34d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.159+0000 7f5f12a00640 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553e328f0 0x556553e34d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80512e5f223453a0 server_cookie=eabe6ab54ba93a3e in_seq=0 out_seq=0 2024-06-02T15:53:29.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.159+0000 7f5efd600640 10 client.26378 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:29.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:29 smithi090 ceph-mon[31595]: pgmap v785: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 281 op/s 2024-06-02T15:53:29.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:29 smithi120 ceph-mon[36451]: pgmap v785: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 281 op/s 2024-06-02T15:53:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:29 smithi135 ceph-mon[36839]: pgmap v785: 129 pgs: 129 active+clean; 293 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 846 KiB/s wr, 281 op/s 2024-06-02T15:53:29.643 INFO:tasks.cephfs.filesystem:scrub status for tag:e47501fb-b9a2-437a-bfa7-181d08fe61df - {'path': '/', 'tag': 'e47501fb-b9a2-437a-bfa7-181d08fe61df', 'options': 'recursive,force'} 2024-06-02T15:53:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.963+0000 7f5efd600640 1 -- 172.21.15.90:0/448954644 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556553d1e2c0 con 0x556553e328f0 2024-06-02T15:53:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.963+0000 7f5efd600640 10 client.26378 handle_command_reply: tid=0 2024-06-02T15:53:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.963+0000 7f5f1d3c4740 1 client.26378 shutdown 2024-06-02T15:53:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 msgr2=0x556553dc3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009c70 tx=0x7f5f040056c0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 shutdown_connections 2024-06-02T15:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556553e328f0 0x556553e34d10 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556553db8690 0x556553dc0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556553db3520 0x556553dc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 --2- 172.21.15.90:0/448954644 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556553db2b30 0x556553dc3cc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 >> 172.21.15.90:0/448954644 conn(0x556553dae1a0 msgr2=0x556553db3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.964+0000 7f5efd600640 0 client.26378 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.966+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 shutdown_connections 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.966+0000 7f5f1d3c4740 1 -- 172.21.15.90:0/448954644 wait complete. 2024-06-02T15:53:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.966+0000 7f5f1d3c4740 20 client.26378 trim_cache size 0 max 16384 2024-06-02T15:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:53:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.967+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 msgr2=0x7f5ef80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.967+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 0x7f5ef80b4eb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x556553db3d80 tx=0x7f5f0c0085f0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 msgr2=0x7f5f14080770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f00031500 tx=0x7f5f00003b90 comp rx=0 tx=0).stop 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 shutdown_connections 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5ef80b29f0 0x7f5ef80b4eb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f1410ac90 0x7f5f141af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f1410a2d0 0x7f5f14080770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 --2- 172.21.15.90:0/948206290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f141090e0 0x7f5f14080230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 >> 172.21.15.90:0/948206290 conn(0x7f5f140780a0 msgr2=0x7f5f1410d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 shutdown_connections 2024-06-02T15:53:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:29.968+0000 7f5f1a600640 1 -- 172.21.15.90:0/948206290 wait complete. 2024-06-02T15:53:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:31 smithi120 ceph-mon[36451]: pgmap v786: 129 pgs: 129 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 413 op/s 2024-06-02T15:53:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:31 smithi135 ceph-mon[36839]: pgmap v786: 129 pgs: 129 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 413 op/s 2024-06-02T15:53:31.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:31 smithi090 ceph-mon[31595]: pgmap v786: 129 pgs: 129 active+clean; 300 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 413 op/s 2024-06-02T15:53:33.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.566 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.567 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.568 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.569 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.570 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: pgmap v787: 129 pgs: 129 active+clean; 301 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 608 op/s 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.571 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.572 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.573 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.574 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.575 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:33.576 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:33 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.579 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.580 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: pgmap v787: 129 pgs: 129 active+clean; 301 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 608 op/s 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:33.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:33 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: pgmap v787: 129 pgs: 129 active+clean; 301 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 608 op/s 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:33.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:33.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:33 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.261 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.261 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.261 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.262 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.263 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.264 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.265 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.266 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.267 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.268 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.269 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.270 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.271 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.272 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.273 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.274 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.275 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.276 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.277 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.278 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.279 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.280 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.281 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.282 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.283 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.284 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.285 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:34.286 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:34 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.576 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.577 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.578 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.581 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.582 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.583 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.584 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.585 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.586 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.587 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.588 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.589 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.590 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.591 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.592 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.593 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:34.594 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:34 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:34.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:34.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:34 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.560 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.561 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.562 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.563 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.564 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.565 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:35 smithi120 ceph-mon[36451]: pgmap v788: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 644 op/s 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.570 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.572 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.573 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.574 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:53:35.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.575 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:35 smithi135 ceph-mon[36839]: pgmap v788: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 644 op/s 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:53:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:53:35.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:35 smithi090 ceph-mon[31595]: pgmap v788: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 644 op/s 2024-06-02T15:53:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:36 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:53:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:36 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:53:36.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:36 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:53:37.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:37 smithi120 ceph-mon[36451]: pgmap v789: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 852 op/s 2024-06-02T15:53:37.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:37 smithi135 ceph-mon[36839]: pgmap v789: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 852 op/s 2024-06-02T15:53:37.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:37 smithi090 ceph-mon[31595]: pgmap v789: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 852 op/s 2024-06-02T15:53:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:53:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:53:38.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:38 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:53:38.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:53:38.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:53:38.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:38 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:53:38.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:53:38.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:53:38.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:38 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:53:39.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:39 smithi120 ceph-mon[36451]: pgmap v790: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 759 op/s 2024-06-02T15:53:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:39 smithi135 ceph-mon[36839]: pgmap v790: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 759 op/s 2024-06-02T15:53:39.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:39 smithi090 ceph-mon[31595]: pgmap v790: 129 pgs: 129 active+clean; 302 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 759 op/s 2024-06-02T15:53:40.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:40 smithi120 ceph-mon[36451]: pgmap v791: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 785 op/s 2024-06-02T15:53:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:40 smithi135 ceph-mon[36839]: pgmap v791: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 785 op/s 2024-06-02T15:53:40.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:40 smithi090 ceph-mon[31595]: pgmap v791: 129 pgs: 129 active+clean; 304 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 785 op/s 2024-06-02T15:53:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.192+0000 7fe347000640 1 -- 172.21.15.90:0/1635813963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3400754b0 msgr2=0x7fe340075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.192+0000 7fe347000640 1 --2- 172.21.15.90:0/1635813963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3400754b0 0x7fe340075930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009900 tx=0x7fe32c02a410 comp rx=0 tx=0).stop 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 -- 172.21.15.90:0/1635813963 shutdown_connections 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 --2- 172.21.15.90:0/1635813963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe34010a7a0 0x7fe34010cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 --2- 172.21.15.90:0/1635813963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3400754b0 0x7fe340075930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 --2- 172.21.15.90:0/1635813963 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe340076a90 0x7fe340074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 -- 172.21.15.90:0/1635813963 >> 172.21.15.90:0/1635813963 conn(0x7fe3400fde50 msgr2=0x7fe340100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 -- 172.21.15.90:0/1635813963 shutdown_connections 2024-06-02T15:53:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.193+0000 7fe347000640 1 -- 172.21.15.90:0/1635813963 wait complete. 2024-06-02T15:53:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.194+0000 7fe347000640 1 Processor -- start 2024-06-02T15:53:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 -- start start 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 0x7fe34019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe34010a7a0 0x7fe3401a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe340110620 con 0x7fe340076a90 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401104a0 con 0x7fe34010a7a0 2024-06-02T15:53:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe347000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401107a0 con 0x7fe3400754b0 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe345c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.195+0000 7fe345c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58756/0 (socket says 172.21.15.90:58756) 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 -- 172.21.15.90:0/16526915 learned_addr learned my addr 172.21.15.90:0/16526915 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345200640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 0x7fe34019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe346600640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe34010a7a0 0x7fe3401a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 -- 172.21.15.90:0/16526915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe34010a7a0 msgr2=0x7fe3401a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe34010a7a0 0x7fe3401a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 -- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 msgr2=0x7fe34019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 0x7fe34019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345c00640 1 -- 172.21.15.90:0/16526915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3401a6d20 con 0x7fe3400754b0 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.196+0000 7fe345200640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 0x7fe34019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe345c00640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe33400ea60 tx=0x7fe33400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe33e000640 1 -- 172.21.15.90:0/16526915 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33400cde0 con 0x7fe3400754b0 2024-06-02T15:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe33e000640 1 -- 172.21.15.90:0/16526915 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3340035f0 con 0x7fe3400754b0 2024-06-02T15:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe33e000640 1 -- 172.21.15.90:0/16526915 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334010690 con 0x7fe3400754b0 2024-06-02T15:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3401a6fb0 con 0x7fe3400754b0 2024-06-02T15:53:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.197+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3401a7420 con 0x7fe3400754b0 2024-06-02T15:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.201+0000 7fe34b423740 1 Processor -- start 2024-06-02T15:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.201+0000 7fe34b423740 1 -- start start 2024-06-02T15:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf56e0 0x55a83ccfa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83ccfcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a83cc64b90 con 0x55a83ccfa850 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a83cc54730 con 0x55a83ccf56e0 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe34b423740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a83cc58770 con 0x55a83ccf4cf0 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe345200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83ccfcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe346600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe345200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83ccfcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37116/0 (socket says 172.21.15.90:37116) 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.202+0000 7fe345200640 1 -- 172.21.15.90:0/3895398325 learned_addr learned my addr 172.21.15.90:0/3895398325 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 -- 172.21.15.90:0/3895398325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf56e0 msgr2=0x55a83ccfa1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf56e0 0x55a83ccfa1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 -- 172.21.15.90:0/3895398325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 msgr2=0x55a83ccfcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83ccfcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 -- 172.21.15.90:0/3895398325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83cbed8b0 con 0x55a83ccf4cf0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe33e000640 1 -- 172.21.15.90:0/16526915 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe3340037a0 con 0x7fe3400754b0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe346600640 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf5110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe338009f00 tx=0x7fe338011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d063b533441eec server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe332a00640 1 -- 172.21.15.90:0/3895398325 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338004d20 con 0x55a83ccf4cf0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe332a00640 1 -- 172.21.15.90:0/3895398325 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338004030 con 0x55a83ccf4cf0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.203+0000 7fe332a00640 1 -- 172.21.15.90:0/3895398325 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338005260 con 0x55a83ccf4cf0 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 -- 172.21.15.90:0/3895398325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 msgr2=0x55a83ccf5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf5110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe338009f00 tx=0x7fe338011810 comp rx=0 tx=0).stop 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 -- 172.21.15.90:0/3895398325 shutdown_connections 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83ccfcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf56e0 0x55a83ccfa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 --2- 172.21.15.90:0/3895398325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf5110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.204+0000 7fe34b423740 1 -- 172.21.15.90:0/3895398325 >> 172.21.15.90:0/3895398325 conn(0x55a83ccf0360 msgr2=0x55a83ccf0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe34b423740 1 -- 172.21.15.90:0/3895398325 shutdown_connections 2024-06-02T15:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe34b423740 1 -- 172.21.15.90:0/3895398325 wait complete. 2024-06-02T15:53:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe33e000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 0x7fe3280b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe33e000640 1 -- 172.21.15.90:0/16526915 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe3340e2910 con 0x7fe3400754b0 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe34b423740 1 -- start start 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe34b423740 1 Processor -- start 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.205+0000 7fe345200640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 0x7fe3280b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83ccfeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83ccff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a83c93a790 con 0x55a83ccfa850 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a83cbfcbc0 con 0x55a83ccf4cf0 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a83ccfd800 con 0x55a83ccf56e0 2024-06-02T15:53:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe34b423740 10 client.0 _ll_register_callbacks cb 0x55a83cbda400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe346600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe346600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37132/0 (socket says 172.21.15.90:37132) 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.206+0000 7fe346600640 1 -- 172.21.15.90:0/2470423318 learned_addr learned my addr 172.21.15.90:0/2470423318 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe345200640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83ccff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 -- 172.21.15.90:0/2470423318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 msgr2=0x55a83ccff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83ccff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 -- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 msgr2=0x55a83ccfeb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83ccfeb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 -- 172.21.15.90:0/2470423318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83c969210 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe346600640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe338004cf0 tx=0x7fe338004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a80acf601eca8a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe327400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.207+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338027020 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe345200640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 0x7fe3280b4cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02a930 tx=0x7fe32c0044e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe338003c30 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe338004030 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe338032050 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe34b423740 10 client.16089 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a83ca35620 con 0x55a83ccfa850 2024-06-02T15:53:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.208+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338004030 con 0x55a83ccfa850 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe338018070 con 0x55a83ccfa850 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 10 client.16089 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 20 client.16089 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd0a440 0x55a83cd0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 4 client.16089 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a83cc66ae0 con 0x55a83cd0a440 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.209+0000 7fe345c00640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd0a440 0x55a83cd0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.210+0000 7fe345c00640 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd0a440 0x55a83cd0c830 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212f412cc60341b3 server_cookie=97ceef59464e63a2 in_seq=0 out_seq=0 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.210+0000 7fe327400640 10 client.16089 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.210+0000 7fe327400640 1 -- 172.21.15.90:0/2470423318 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a83cc66ae0 con 0x55a83cd0a440 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.210+0000 7fe327400640 10 client.16089 handle_command_reply: tid=0 2024-06-02T15:53:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.210+0000 7fe34b423740 1 client.16089 shutdown 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 msgr2=0x55a83cd060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe338004cf0 tx=0x7fe338004d20 comp rx=0 tx=0).stop 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 shutdown_connections 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd0a440 0x55a83cd0c830 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd060b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83ccff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 --2- 172.21.15.90:0/2470423318 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83ccfeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 >> 172.21.15.90:0/2470423318 conn(0x55a83ccf0360 msgr2=0x55a83ccef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe327400640 0 client.16089 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 shutdown_connections 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 1 -- 172.21.15.90:0/2470423318 wait complete. 2024-06-02T15:53:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.211+0000 7fe34b423740 20 client.16089 trim_cache size 0 max 16384 2024-06-02T15:53:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.230+0000 7fe34b423740 1 Processor -- start 2024-06-02T15:53:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 -- start start 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 0x55a83cd06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 0x55a83cd11130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b44c0 con 0x55a83ccf56e0 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b1cd0 con 0x55a83ccfa850 2024-06-02T15:53:43.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe34b423740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401afb20 con 0x55a83ccf4cf0 2024-06-02T15:53:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe345c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe345c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58778/0 (socket says 172.21.15.90:58778) 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe346600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 0x55a83cd11130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe345c00640 1 -- 172.21.15.90:0/2433295996 learned_addr learned my addr 172.21.15.90:0/2433295996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.231+0000 7fe345200640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 0x55a83cd06070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 -- 172.21.15.90:0/2433295996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 msgr2=0x55a83ccf6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 -- 172.21.15.90:0/2433295996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 msgr2=0x55a83cd06070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 0x55a83cd06070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 -- 172.21.15.90:0/2433295996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83ccf4930 con 0x55a83ccfa850 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe345c00640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe345200640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 0x55a83cd06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe346600640 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 0x55a83cd11130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe338011850 tx=0x7fe338005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628cf32b9d2f3a68 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe332000640 1 -- 172.21.15.90:0/2433295996 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338027070 con 0x55a83ccfa850 2024-06-02T15:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe332000640 1 -- 172.21.15.90:0/2433295996 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338004d90 con 0x55a83ccfa850 2024-06-02T15:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.232+0000 7fe332000640 1 -- 172.21.15.90:0/2433295996 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33801e040 con 0x55a83ccfa850 2024-06-02T15:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 -- 172.21.15.90:0/2433295996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 msgr2=0x55a83cd11130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 0x55a83cd11130 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe338011850 tx=0x7fe338005f50 comp rx=0 tx=0).stop 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 -- 172.21.15.90:0/2433295996 shutdown_connections 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccfa850 0x55a83cd11130 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccf56e0 0x55a83cd06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.233+0000 7fe34b423740 1 --2- 172.21.15.90:0/2433295996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf4cf0 0x55a83ccf6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.234+0000 7fe34b423740 1 -- 172.21.15.90:0/2433295996 >> 172.21.15.90:0/2433295996 conn(0x55a83ccf0360 msgr2=0x55a83ccf38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.234+0000 7fe34b423740 1 -- 172.21.15.90:0/2433295996 shutdown_connections 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.234+0000 7fe34b423740 1 -- 172.21.15.90:0/2433295996 wait complete. 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.234+0000 7fe34b423740 1 -- start start 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 Processor -- start 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83cd03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83cd03cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd04c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b0190 con 0x55a83ccfa850 2024-06-02T15:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b1600 con 0x55a83ccf4cf0 2024-06-02T15:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3401b0860 con 0x55a83ccf56e0 2024-06-02T15:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.235+0000 7fe34b423740 10 client.0 _ll_register_callbacks cb 0x7fe334005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83cd03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83cd03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83cd03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58794/0 (socket says 172.21.15.90:58794) 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345200640 1 -- 172.21.15.90:0/3692349997 learned_addr learned my addr 172.21.15.90:0/3692349997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 -- 172.21.15.90:0/3692349997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 msgr2=0x55a83cd03cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe346600640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd04c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83cd03cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 -- 172.21.15.90:0/3692349997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 msgr2=0x55a83cd04c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd04c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 -- 172.21.15.90:0/3692349997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83cd05250 con 0x55a83ccf4cf0 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe346600640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd04c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe345c00640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83cd03780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c760 tx=0x7fe334003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a10b5291588b53d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.236+0000 7fe327400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.237+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33408e030 con 0x55a83ccf4cf0 2024-06-02T15:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.237+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe3340a6110 con 0x55a83ccf4cf0 2024-06-02T15:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.237+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe338004d90 con 0x55a83ccf4cf0 2024-06-02T15:53:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.239+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe3340a9630 con 0x55a83ccf4cf0 2024-06-02T15:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.239+0000 7fe34b423740 10 client.26390 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.239+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a83cd75010 con 0x55a83ccf4cf0 2024-06-02T15:53:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.239+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe338004d90 con 0x55a83ccf4cf0 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe334004000 con 0x55a83ccf4cf0 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 10 client.26390 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 20 client.26390 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd75940 0x55a83cd77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 4 client.26390 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:53:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a83cc602c0 con 0x55a83cd75940 2024-06-02T15:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.240+0000 7fe345200640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd75940 0x55a83cd77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.241+0000 7fe345200640 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd75940 0x55a83cd77d60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3c1e1060c6864e server_cookie=256b6c34496c8cb6 in_seq=0 out_seq=0 2024-06-02T15:53:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.241+0000 7fe327400640 10 client.26390 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe327400640 1 -- 172.21.15.90:0/3692349997 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a83cc602c0 con 0x55a83cd75940 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe327400640 10 client.26390 handle_command_reply: tid=0 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 client.26390 shutdown 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 msgr2=0x55a83cd03780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83cd03780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c760 tx=0x7fe334003e90 comp rx=0 tx=0).stop 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 shutdown_connections 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55a83cd75940 0x55a83cd77d60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a83ccfa850 0x55a83cd04c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a83ccf56e0 0x55a83cd03cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 --2- 172.21.15.90:0/3692349997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a83ccf4cf0 0x55a83cd03780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 >> 172.21.15.90:0/3692349997 conn(0x55a83ccf0360 msgr2=0x55a83ccef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe327400640 0 client.26390 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 shutdown_connections 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.264+0000 7fe34b423740 1 -- 172.21.15.90:0/3692349997 wait complete. 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.265+0000 7fe34b423740 20 client.26390 trim_cache size 0 max 16384 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 msgr2=0x7fe3280b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 0x7fe3280b4cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe32c02a930 tx=0x7fe32c0044e0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 msgr2=0x7fe34019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe33400ea60 tx=0x7fe33400ef20 comp rx=0 tx=0).stop 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 shutdown_connections 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe3280b27f0 0x7fe3280b4cb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe34010a7a0 0x7fe3401a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe340076a90 0x7fe34019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 --2- 172.21.15.90:0/16526915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3400754b0 0x7fe34019f150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 >> 172.21.15.90:0/16526915 conn(0x7fe3400fde50 msgr2=0x7fe340102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 shutdown_connections 2024-06-02T15:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:43.266+0000 7fe347000640 1 -- 172.21.15.90:0/16526915 wait complete. 2024-06-02T15:53:43.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:43 smithi090 ceph-mon[31595]: pgmap v792: 129 pgs: 129 active+clean; 307 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 702 op/s 2024-06-02T15:53:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:43 smithi120 ceph-mon[36451]: pgmap v792: 129 pgs: 129 active+clean; 307 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 702 op/s 2024-06-02T15:53:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:43 smithi135 ceph-mon[36839]: pgmap v792: 129 pgs: 129 active+clean; 307 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 702 op/s 2024-06-02T15:53:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:45 smithi120 ceph-mon[36451]: pgmap v793: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 959 KiB/s wr, 509 op/s 2024-06-02T15:53:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:45 smithi135 ceph-mon[36839]: pgmap v793: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 959 KiB/s wr, 509 op/s 2024-06-02T15:53:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:45 smithi090 ceph-mon[31595]: pgmap v793: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 959 KiB/s wr, 509 op/s 2024-06-02T15:53:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:47 smithi120 ceph-mon[36451]: pgmap v794: 129 pgs: 129 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 506 op/s 2024-06-02T15:53:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:47 smithi135 ceph-mon[36839]: pgmap v794: 129 pgs: 129 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 506 op/s 2024-06-02T15:53:47.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:47 smithi090 ceph-mon[31595]: pgmap v794: 129 pgs: 129 active+clean; 306 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 506 op/s 2024-06-02T15:53:48.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:48 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:53:48.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:48 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:53:48.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:48 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:53:48.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:48 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:53:48.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:48 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:53:48.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:48 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:53:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:49 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:53:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:49 smithi120 ceph-mon[36451]: pgmap v795: 129 pgs: 129 active+clean; 306 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 745 KiB/s wr, 283 op/s 2024-06-02T15:53:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:49 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:53:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:49 smithi135 ceph-mon[36839]: pgmap v795: 129 pgs: 129 active+clean; 306 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 745 KiB/s wr, 283 op/s 2024-06-02T15:53:49.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:49 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:53:49.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:49 smithi090 ceph-mon[31595]: pgmap v795: 129 pgs: 129 active+clean; 306 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 745 KiB/s wr, 283 op/s 2024-06-02T15:53:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:51 smithi120 ceph-mon[36451]: pgmap v796: 129 pgs: 129 active+clean; 308 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 833 KiB/s wr, 266 op/s 2024-06-02T15:53:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:51 smithi135 ceph-mon[36839]: pgmap v796: 129 pgs: 129 active+clean; 308 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 833 KiB/s wr, 266 op/s 2024-06-02T15:53:51.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:51 smithi090 ceph-mon[31595]: pgmap v796: 129 pgs: 129 active+clean; 308 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 833 KiB/s wr, 266 op/s 2024-06-02T15:53:53.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:53 smithi120 ceph-mon[36451]: pgmap v797: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 737 KiB/s wr, 242 op/s 2024-06-02T15:53:53.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:53 smithi135 ceph-mon[36839]: pgmap v797: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 737 KiB/s wr, 242 op/s 2024-06-02T15:53:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:53 smithi090 ceph-mon[31595]: pgmap v797: 129 pgs: 129 active+clean; 308 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 737 KiB/s wr, 242 op/s 2024-06-02T15:53:53.856 INFO:tasks.workunit.client.0.smithi090.stdout:cleaned up hierarchy 2024-06-02T15:53:54.280 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:53:54.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:53:54.367 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2024-06-02T15:53:54.367 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2024-06-02T15:53:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:55 smithi120 ceph-mon[36451]: pgmap v798: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 793 KiB/s wr, 224 op/s 2024-06-02T15:53:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:55 smithi135 ceph-mon[36839]: pgmap v798: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 793 KiB/s wr, 224 op/s 2024-06-02T15:53:55.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:55 smithi090 ceph-mon[31595]: pgmap v798: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 793 KiB/s wr, 224 op/s 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 -- 172.21.15.90:0/3834883614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0104cd0 msgr2=0x7f85a01050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 --2- 172.21.15.90:0/3834883614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0104cd0 0x7f85a01050f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402a3f0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 -- 172.21.15.90:0/3834883614 shutdown_connections 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 --2- 172.21.15.90:0/3834883614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0106880 0x7f85a010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 --2- 172.21.15.90:0/3834883614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0105ec0 0x7f85a0106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 --2- 172.21.15.90:0/3834883614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0104cd0 0x7f85a01050f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 -- 172.21.15.90:0/3834883614 >> 172.21.15.90:0/3834883614 conn(0x7f85a0100240 msgr2=0x7f85a0102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 -- 172.21.15.90:0/3834883614 shutdown_connections 2024-06-02T15:53:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.408+0000 7f85a7e00640 1 -- 172.21.15.90:0/3834883614 wait complete. 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.409+0000 7f85a7e00640 1 Processor -- start 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.409+0000 7f85a7e00640 1 -- start start 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 0x7f85a019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 0x7f85a019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0106880 0x7f85a01a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110850 con 0x7f85a0104cd0 2024-06-02T15:53:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01106d0 con 0x7f85a0106880 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a7e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01109d0 con 0x7f85a0105ec0 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 0x7f85a019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 0x7f85a019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52290/0 (socket says 172.21.15.90:52290) 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a4a00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 0x7f85a019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a5400640 1 -- 172.21.15.90:0/961398755 learned_addr learned my addr 172.21.15.90:0/961398755 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a5e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0106880 0x7f85a01a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a4a00640 1 -- 172.21.15.90:0/961398755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0106880 msgr2=0x7f85a01a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.410+0000 7f85a4a00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0106880 0x7f85a01a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a4a00640 1 -- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 msgr2=0x7f85a019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a4a00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 0x7f85a019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a4a00640 1 -- 172.21.15.90:0/961398755 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a01a2a50 con 0x7f85a0105ec0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a4a00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 0x7f85a019b370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e9f0 tx=0x7f858c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f859e000640 1 -- 172.21.15.90:0/961398755 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c00ccc0 con 0x7f85a0105ec0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a2d40 con 0x7f85a0105ec0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f859e000640 1 -- 172.21.15.90:0/961398755 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858c0108e0 con 0x7f85a0105ec0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f859e000640 1 -- 172.21.15.90:0/961398755 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c018880 con 0x7f85a0105ec0 2024-06-02T15:53:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.411+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a3230 con 0x7f85a0105ec0 2024-06-02T15:53:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 Processor -- start 2024-06-02T15:53:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 -- start start 2024-06-02T15:53:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f859e000640 1 -- 172.21.15.90:0/961398755 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f858c010430 con 0x7f85a0105ec0 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec94b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55edfebfcb90 con 0x55edfec92690 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55edfebec730 con 0x55edfec8d520 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85aad8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edfebf0770 con 0x55edfec8cb30 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.415+0000 7f85a5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43326/0 (socket says 172.21.15.90:43326) 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec94b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 -- 172.21.15.90:0/1068529213 learned_addr learned my addr 172.21.15.90:0/1068529213 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 -- 172.21.15.90:0/1068529213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 msgr2=0x55edfec8cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 -- 172.21.15.90:0/1068529213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 msgr2=0x55edfec94b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec94b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 -- 172.21.15.90:0/1068529213 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfeb858b0 con 0x55edfec8d520 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a4a00640 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec94b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f85a5400640 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f859402a9e0 tx=0x7f8594004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7b5842cfd28cae server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.416+0000 7f8592600640 1 -- 172.21.15.90:0/1068529213 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594035030 con 0x55edfec8d520 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f8592600640 1 -- 172.21.15.90:0/1068529213 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8594040070 con 0x55edfec8d520 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f8592600640 1 -- 172.21.15.90:0/1068529213 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594031960 con 0x55edfec8d520 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f859e000640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 0x7f85880b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f859e000640 1 -- 172.21.15.90:0/961398755 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f858c0e28e0 con 0x7f85a0105ec0 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f85a5e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 0x7f85880b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f85aad8c740 1 -- 172.21.15.90:0/1068529213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 msgr2=0x55edfec92020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.417+0000 7f85aad8c740 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f859402a9e0 tx=0x7f8594004040 comp rx=0 tx=0).stop 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 -- 172.21.15.90:0/1068529213 shutdown_connections 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec94b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec92020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 --2- 172.21.15.90:0/1068529213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 -- 172.21.15.90:0/1068529213 >> 172.21.15.90:0/1068529213 conn(0x55edfec881a0 msgr2=0x55edfec885a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 -- 172.21.15.90:0/1068529213 shutdown_connections 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85a5e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 0x7f85880b4d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55edfec8dd80 tx=0x7f8598009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 -- 172.21.15.90:0/1068529213 wait complete. 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 -- start start 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.418+0000 7f85aad8c740 1 Processor -- start 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec9de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55edfe8d2790 con 0x55edfec92690 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55edfeb94bc0 con 0x55edfec8d520 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edfec955e0 con 0x55edfec8cb30 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85aad8c740 10 client.0 _ll_register_callbacks cb 0x55edfeb72400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:32850/0 (socket says 172.21.15.90:32850) 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a5400640 1 -- 172.21.15.90:0/4143763961 learned_addr learned my addr 172.21.15.90:0/4143763961 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43330/0 (socket says 172.21.15.90:43330) 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.419+0000 7f85a5e00640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec9de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 -- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 msgr2=0x55edfec96e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 -- 172.21.15.90:0/4143763961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 msgr2=0x55edfec9de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec9de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 -- 172.21.15.90:0/4143763961 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfe901210 con 0x55edfec8cb30 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a4a00640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5e00640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec9de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85a5400640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8594006010 tx=0x7f8594009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb14610401bd627 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f8586a00640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594052060 con 0x55edfec8cb30 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8594040070 con 0x55edfec8cb30 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.420+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8594003a00 con 0x55edfec8cb30 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8594059050 con 0x55edfec8cb30 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 10 client.26338 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55edfe9cd620 con 0x55edfec8cb30 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8594040070 con 0x55edfec8cb30 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f8594045600 con 0x55edfec8cb30 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 10 client.26338 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 20 client.26338 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfeca1fe0 0x55edfeca43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 4 client.26338 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:53:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.421+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edfebfeae0 con 0x55edfeca1fe0 2024-06-02T15:53:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.422+0000 7f85a4a00640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfeca1fe0 0x55edfeca43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.422+0000 7f85a4a00640 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfeca1fe0 0x55edfeca43d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f1ed84867c3a8b server_cookie=baeb982b96fb4da5 in_seq=0 out_seq=0 2024-06-02T15:53:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.422+0000 7f8586a00640 10 client.26338 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.423+0000 7f8586a00640 1 -- 172.21.15.90:0/4143763961 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55edfebfeae0 con 0x55edfeca1fe0 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.423+0000 7f8586a00640 10 client.26338 handle_command_reply: tid=0 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.423+0000 7f85aad8c740 1 client.26338 shutdown 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.423+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 msgr2=0x55edfec96900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.423+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8594006010 tx=0x7f8594009260 comp rx=0 tx=0).stop 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 shutdown_connections 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfeca1fe0 0x55edfeca43d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec92690 0x55edfec9de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec96e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f8586a00640 0 client.26338 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 --2- 172.21.15.90:0/4143763961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec96900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 >> 172.21.15.90:0/4143763961 conn(0x55edfec881a0 msgr2=0x55edfec87310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 shutdown_connections 2024-06-02T15:53:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 1 -- 172.21.15.90:0/4143763961 wait complete. 2024-06-02T15:53:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.424+0000 7f85aad8c740 20 client.26338 trim_cache size 0 max 16384 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 Processor -- start 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 -- start start 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01b0140 con 0x55edfec8d520 2024-06-02T15:53:56.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ad950 con 0x55edfec92690 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85aad8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ab7a0 con 0x55edfec8cb30 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85a5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.442+0000 7f85a5e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43342/0 (socket says 172.21.15.90:43342) 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52314/0 (socket says 172.21.15.90:52314) 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 -- 172.21.15.90:0/3783623409 learned_addr learned my addr 172.21.15.90:0/3783623409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 -- 172.21.15.90:0/3783623409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 msgr2=0x55edfec8ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 -- 172.21.15.90:0/3783623409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 msgr2=0x55edfec9de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 -- 172.21.15.90:0/3783623409 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfec87a20 con 0x55edfec92690 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5400640 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a4a00640 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.443+0000 7f85a5e00640 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85980099c0 tx=0x7f8598016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d6cd508528ea71 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f8591c00640 1 -- 172.21.15.90:0/3783623409 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980045c0 con 0x55edfec92690 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f8591c00640 1 -- 172.21.15.90:0/3783623409 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8598004760 con 0x55edfec92690 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f8591c00640 1 -- 172.21.15.90:0/3783623409 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859801e5f0 con 0x55edfec92690 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 -- 172.21.15.90:0/3783623409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 msgr2=0x55edfec95f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f85980099c0 tx=0x7f8598016000 comp rx=0 tx=0).stop 2024-06-02T15:53:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 -- 172.21.15.90:0/3783623409 shutdown_connections 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec92690 0x55edfec95f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8d520 0x55edfec9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 --2- 172.21.15.90:0/3783623409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec8cb30 0x55edfec8ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.444+0000 7f85aad8c740 1 -- 172.21.15.90:0/3783623409 >> 172.21.15.90:0/3783623409 conn(0x55edfec881a0 msgr2=0x55edfec87310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.445+0000 7f85aad8c740 1 -- 172.21.15.90:0/3783623409 shutdown_connections 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.445+0000 7f85aad8c740 1 -- 172.21.15.90:0/3783623409 wait complete. 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.445+0000 7f85aad8c740 1 -- start start 2024-06-02T15:53:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.445+0000 7f85aad8c740 1 Processor -- start 2024-06-02T15:53:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec92690 0x55edfed0aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01abe10 con 0x55edfec8cb30 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ad280 con 0x55edfec8d520 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85a01ac4e0 con 0x55edfec92690 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85aad8c740 10 client.0 _ll_register_callbacks cb 0x7f8598009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52324/0 (socket says 172.21.15.90:52324) 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 -- 172.21.15.90:0/626149073 learned_addr learned my addr 172.21.15.90:0/626149073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a4a00640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5e00640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec92690 0x55edfed0aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 -- 172.21.15.90:0/626149073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec92690 msgr2=0x55edfed0aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec92690 0x55edfed0aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 -- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 msgr2=0x55edfec9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec9af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.446+0000 7f85a5400640 1 -- 172.21.15.90:0/626149073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfec9dfa0 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f85a5400640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55edfec86770 tx=0x7f8594045ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4847e4ceaedf1fee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f8586a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594052040 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f8594003800 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8598004760 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8594066050 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f85aad8c740 10 client.16110 fetch_fsmap learned FSMap version 37 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.447+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55edfed0bf30 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8598004760 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f85940318f0 con 0x55edfec8cb30 2024-06-02T15:53:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 10 client.16110 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 20 client.16110 populate_metadata read hostname 'smithi090' 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfed0c960 0x55edfed0ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 4 client.16110 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edfebf82c0 con 0x55edfed0c960 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.448+0000 7f85a4a00640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfed0c960 0x55edfed0ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.449+0000 7f85a4a00640 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfed0c960 0x55edfed0ed80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38948de25fb6f85 server_cookie=d3812c0b3ce40744 in_seq=0 out_seq=0 2024-06-02T15:53:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.449+0000 7f8586a00640 10 client.16110 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f8586a00640 1 -- 172.21.15.90:0/626149073 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55edfebf82c0 con 0x55edfed0c960 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f8586a00640 10 client.16110 handle_command_reply: tid=0 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 client.16110 shutdown 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 msgr2=0x55edfec9aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55edfec86770 tx=0x7f8594045ea0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 shutdown_connections 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55edfed0c960 0x55edfed0ed80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfec92690 0x55edfed0aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55edfec8d520 0x55edfec9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 --2- 172.21.15.90:0/626149073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edfec8cb30 0x55edfec9aa30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f8586a00640 0 client.16110 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 >> 172.21.15.90:0/626149073 conn(0x55edfec881a0 msgr2=0x55edfec8ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 shutdown_connections 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.467+0000 7f85aad8c740 1 -- 172.21.15.90:0/626149073 wait complete. 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85aad8c740 20 client.16110 trim_cache size 0 max 16384 2024-06-02T15:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 msgr2=0x7f85880b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 0x7f85880b4d60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55edfec8dd80 tx=0x7f8598009500 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 msgr2=0x7f85a019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 0x7f85a019b370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f858c00e9f0 tx=0x7f858c00eeb0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 shutdown_connections 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f85880b28a0 0x7f85880b4d60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f85a0106880 0x7f85a01a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85a0105ec0 0x7f85a019b370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 --2- 172.21.15.90:0/961398755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85a0104cd0 0x7f85a019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.468+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 >> 172.21.15.90:0/961398755 conn(0x7f85a0100240 msgr2=0x7f85a0109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.469+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 shutdown_connections 2024-06-02T15:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:53:56.469+0000 7f85a7e00640 1 -- 172.21.15.90:0/961398755 wait complete. 2024-06-02T15:53:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:57 smithi120 ceph-mon[36451]: pgmap v799: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 858 KiB/s wr, 243 op/s 2024-06-02T15:53:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:57 smithi135 ceph-mon[36839]: pgmap v799: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 858 KiB/s wr, 243 op/s 2024-06-02T15:53:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:57 smithi090 ceph-mon[31595]: pgmap v799: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 858 KiB/s wr, 243 op/s 2024-06-02T15:53:58.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:53:58 smithi120 ceph-mon[36451]: pgmap v800: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 625 KiB/s wr, 165 op/s 2024-06-02T15:53:58.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:53:58 smithi135 ceph-mon[36839]: pgmap v800: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 625 KiB/s wr, 165 op/s 2024-06-02T15:53:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:53:58 smithi090 ceph-mon[31595]: pgmap v800: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 625 KiB/s wr, 165 op/s 2024-06-02T15:53:59.645 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:54:00.077 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:01.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:01 smithi120 ceph-mon[36451]: pgmap v801: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 152 op/s 2024-06-02T15:54:01.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:01 smithi135 ceph-mon[36839]: pgmap v801: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 152 op/s 2024-06-02T15:54:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:01 smithi090 ceph-mon[31595]: pgmap v801: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 623 KiB/s wr, 152 op/s 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 -- 172.21.15.90:0/148652684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc075970 msgr2=0x7fe0fc075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 --2- 172.21.15.90:0/148652684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc075970 0x7fe0fc075d50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009990 tx=0x7fe0f0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 -- 172.21.15.90:0/148652684 shutdown_connections 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 --2- 172.21.15.90:0/148652684 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 --2- 172.21.15.90:0/148652684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc075970 0x7fe0fc075d50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 --2- 172.21.15.90:0/148652684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 0x7fe0fc10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.452+0000 7fe102000700 1 -- 172.21.15.90:0/148652684 >> 172.21.15.90:0/148652684 conn(0x7fe0fc0fefc0 msgr2=0x7fe0fc1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.453+0000 7fe102000700 1 -- 172.21.15.90:0/148652684 shutdown_connections 2024-06-02T15:54:02.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.453+0000 7fe102000700 1 -- 172.21.15.90:0/148652684 wait complete. 2024-06-02T15:54:02.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 Processor -- start 2024-06-02T15:54:02.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 -- start start 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 0x7fe0fc19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1158b0 con 0x7fe0fc10fcb0 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc115730 con 0x7fe0fc075970 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.454+0000 7fe102000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc115a30 con 0x7fe0fc076290 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe0fbe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 0x7fe0fc19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:55188/0 (socket says 172.21.15.90:55188) 2024-06-02T15:54:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50302/0 (socket says 172.21.15.90:50302) 2024-06-02T15:54:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 -- 172.21.15.90:0/3602974661 learned_addr learned my addr 172.21.15.90:0/3602974661 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 -- 172.21.15.90:0/3602974661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 msgr2=0x7fe0fc19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 0x7fe0fc19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 -- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 msgr2=0x7fe0fc1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 -- 172.21.15.90:0/3602974661 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1a0db0 con 0x7fe0fc075970 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe101600700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.455+0000 7fe100c00700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.456+0000 7fe0f9600700 1 -- 172.21.15.90:0/3602974661 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0027070 con 0x7fe0fc075970 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.456+0000 7fe0f9600700 1 -- 172.21.15.90:0/3602974661 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f0022470 con 0x7fe0fc075970 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.456+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1a1040 con 0x7fe0fc075970 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.456+0000 7fe0f9600700 1 -- 172.21.15.90:0/3602974661 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0017650 con 0x7fe0fc075970 2024-06-02T15:54:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.456+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a8920 con 0x7fe0fc075970 2024-06-02T15:54:02.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.460+0000 7fe0f9600700 1 -- 172.21.15.90:0/3602974661 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe0f002a070 con 0x7fe0fc075970 2024-06-02T15:54:02.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 Processor -- start 2024-06-02T15:54:02.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 -- start start 2024-06-02T15:54:02.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe0f9600700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 0x7fe0e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe0f9600700 1 -- 172.21.15.90:0/3602974661 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0f00e32b0 con 0x7fe0fc075970 2024-06-02T15:54:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b940 0x55bc2df754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bc2ded3e40 con 0x55bc2df75ad0 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc2dec39e0 con 0x55bc2df7b940 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe10e063b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc2dec7a20 con 0x55bc2df7b020 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.461+0000 7fe101600700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 0x7fe0e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44330/0 (socket says 172.21.15.90:44330) 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 -- 172.21.15.90:0/1163767624 learned_addr learned my addr 172.21.15.90:0/1163767624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b940 0x55bc2df754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe101600700 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 -- 172.21.15.90:0/1163767624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b940 msgr2=0x55bc2df754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b940 0x55bc2df754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 -- 172.21.15.90:0/1163767624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 msgr2=0x55bc2df75f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df75f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 -- 172.21.15.90:0/1163767624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc2de8c420 con 0x55bc2df7b020 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.462+0000 7fe100c00700 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005cb0 tx=0x7fe0f00aa3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60aadf4d61aa9ea8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe101600700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 0x7fe0e40b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009fd0 tx=0x7fe0f4007b00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe0ee000700 1 -- 172.21.15.90:0/1163767624 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00e0e60 con 0x55bc2df7b020 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe0ee000700 1 -- 172.21.15.90:0/1163767624 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00aac50 con 0x55bc2df7b020 2024-06-02T15:54:02.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe0ee000700 1 -- 172.21.15.90:0/1163767624 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0022940 con 0x55bc2df7b020 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe10e063b80 1 -- 172.21.15.90:0/1163767624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 msgr2=0x55bc2df7b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.463+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0005cb0 tx=0x7fe0f00aa3a0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 -- 172.21.15.90:0/1163767624 shutdown_connections 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df75f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df7b400 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1163767624 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b940 0x55bc2df754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 -- 172.21.15.90:0/1163767624 >> 172.21.15.90:0/1163767624 conn(0x55bc2dea3be0 msgr2=0x55bc2df70150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 -- 172.21.15.90:0/1163767624 shutdown_connections 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 -- 172.21.15.90:0/1163767624 wait complete. 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 -- start start 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.464+0000 7fe10e063b80 1 Processor -- start 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 0x55bc2df7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bc2ddc9fa0 con 0x55bc2df75ad0 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bc2de64760 con 0x55bc2df7b020 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc2df82220 con 0x55bc2df7b940 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe10e063b80 10 client.0 _ll_register_callbacks cb 0x55bc2de6dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe100c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe100c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50324/0 (socket says 172.21.15.90:50324) 2024-06-02T15:54:02.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe0fbe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 0x55bc2df7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe100c00700 1 -- 172.21.15.90:0/1504380394 learned_addr learned my addr 172.21.15.90:0/1504380394 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.465+0000 7fe101600700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 -- 172.21.15.90:0/1504380394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 msgr2=0x55bc2df7ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df7ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 -- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 msgr2=0x55bc2df7ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 0x55bc2df7ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 -- 172.21.15.90:0/1504380394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2de138d0 con 0x55bc2df75ad0 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 0x55bc2df7ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe101600700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe100c00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00aa250 tx=0x7fe0f0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafb7318189956c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe0e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:02.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00c4060 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0f00b2e90 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.466+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f00aac50 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe0f00e0e60 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 10 client.16122 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bc2de94060 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00aac50 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe0f00cb020 con 0x55bc2df75ad0 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 20 client.16122 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2df8eb20 0x55bc2df90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 4 client.16122 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:02.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.467+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc2ded5d90 con 0x55bc2df8eb20 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.468+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2df8eb20 0x55bc2df90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.468+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2df8eb20 0x55bc2df90f10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b509502e18bb7e2f server_cookie=a1b5514e07071d9c in_seq=0 out_seq=0 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.468+0000 7fe0e2a00700 10 client.16122 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.469+0000 7fe0e2a00700 1 -- 172.21.15.90:0/1504380394 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bc2ded5d90 con 0x55bc2df8eb20 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.469+0000 7fe0e2a00700 10 client.16122 handle_command_reply: tid=0 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.469+0000 7fe10e063b80 1 client.16122 shutdown 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 msgr2=0x55bc2df815f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00aa250 tx=0x7fe0f0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 shutdown_connections 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2df8eb20 0x55bc2df90f10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df75ad0 0x55bc2df815f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 --2- 172.21.15.90:0/1504380394 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df7b020 0x55bc2df7ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 >> 172.21.15.90:0/1504380394 conn(0x55bc2dea3be0 msgr2=0x55bc2df744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe0e2a00700 0 client.16122 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 shutdown_connections 2024-06-02T15:54:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 1 -- 172.21.15.90:0/1504380394 wait complete. 2024-06-02T15:54:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.470+0000 7fe10e063b80 20 client.16122 trim_cache size 0 max 16384 2024-06-02T15:54:02.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.492+0000 7fe10e063b80 1 Processor -- start 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 -- start start 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 0x55bc2df80e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df6fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b3d10 con 0x55bc2df7b020 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b14e0 con 0x55bc2df75ad0 2024-06-02T15:54:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe10e063b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1af330 con 0x55bc2df7b940 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe0fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 0x55bc2df80e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe0fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 0x55bc2df80e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50326/0 (socket says 172.21.15.90:50326) 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe0fbe00700 1 -- 172.21.15.90:0/2860034289 learned_addr learned my addr 172.21.15.90:0/2860034289 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe100c00700 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.493+0000 7fe101600700 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df6fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 -- 172.21.15.90:0/2860034289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 msgr2=0x55bc2df6fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df6fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 -- 172.21.15.90:0/2860034289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 msgr2=0x55bc2df80e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 0x55bc2df80e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 -- 172.21.15.90:0/2860034289 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc2de94060 con 0x55bc2df75ad0 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe100c00700 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df754b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009750 tx=0x7fe0f00aa3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cab59d758b2ca0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe0ed600700 1 -- 172.21.15.90:0/2860034289 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00c4070 con 0x55bc2df75ad0 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe0ed600700 1 -- 172.21.15.90:0/2860034289 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00a5ec0 con 0x55bc2df75ad0 2024-06-02T15:54:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.494+0000 7fe0ed600700 1 -- 172.21.15.90:0/2860034289 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00aa550 con 0x55bc2df75ad0 2024-06-02T15:54:02.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 -- 172.21.15.90:0/2860034289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 msgr2=0x55bc2df754b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df754b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009750 tx=0x7fe0f00aa3a0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 -- 172.21.15.90:0/2860034289 shutdown_connections 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b940 0x55bc2df6fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df754b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 --2- 172.21.15.90:0/2860034289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b020 0x55bc2df80e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 -- 172.21.15.90:0/2860034289 >> 172.21.15.90:0/2860034289 conn(0x55bc2dea3be0 msgr2=0x55bc2df78b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 -- 172.21.15.90:0/2860034289 shutdown_connections 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.495+0000 7fe10e063b80 1 -- 172.21.15.90:0/2860034289 wait complete. 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 -- start start 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 Processor -- start 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b940 0x55bc2df84440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1af9a0 con 0x55bc2df7b940 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b0e10 con 0x55bc2df75ad0 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b0070 con 0x55bc2df7b020 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.496+0000 7fe10e063b80 10 client.0 _ll_register_callbacks cb 0x55bc2de6dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe101600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b940 0x55bc2df84440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:55232/0 (socket says 172.21.15.90:55232) 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 -- 172.21.15.90:0/347617071 learned_addr learned my addr 172.21.15.90:0/347617071 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 -- 172.21.15.90:0/347617071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 msgr2=0x55bc2df879b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df879b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 -- 172.21.15.90:0/347617071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b940 msgr2=0x55bc2df84440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b940 0x55bc2df84440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 -- 172.21.15.90:0/347617071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc2df84980 con 0x55bc2df75ad0 2024-06-02T15:54:02.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe100c00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00aae00 tx=0x7fe0f0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5caa8cae0f4b1a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.497+0000 7fe0e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00c4070 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f00a5ec0 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fe0f00b2e90 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe0f00053d0 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 10 client.26426 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bc2df84980 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00a5ec0 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe0f00cb050 con 0x55bc2df75ad0 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 10 client.26426 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 10 client.26426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:02.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 20 client.26426 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2dff9ca0 0x55bc2dffc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 4 client.26426 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.498+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc2decf570 con 0x55bc2dff9ca0 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2dff9ca0 0x55bc2dffc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe0fbe00700 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2dff9ca0 0x55bc2dffc0c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ded3cb46cc6969 server_cookie=aa0e0c11583f2d2f in_seq=0 out_seq=0 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe0e2a00700 10 client.26426 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe0e2a00700 1 -- 172.21.15.90:0/347617071 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc2decf570 con 0x55bc2dff9ca0 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe0e2a00700 10 client.26426 handle_command_reply: tid=0 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.499+0000 7fe10e063b80 1 client.26426 shutdown 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 msgr2=0x55bc2df87470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00aae00 tx=0x7fe0f0005fd0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 shutdown_connections 2024-06-02T15:54:02.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bc2dff9ca0 0x55bc2dffc0c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bc2df7b940 0x55bc2df84440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe0e2a00700 0 client.26426 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bc2df75ad0 0x55bc2df87470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 --2- 172.21.15.90:0/347617071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc2df7b020 0x55bc2df879b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 >> 172.21.15.90:0/347617071 conn(0x55bc2dea3be0 msgr2=0x55bc2df7d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 shutdown_connections 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 1 -- 172.21.15.90:0/347617071 wait complete. 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.500+0000 7fe10e063b80 20 client.26426 trim_cache size 0 max 16384 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:54:02.500 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 msgr2=0x7fe0e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 0x7fe0e40b4d40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009fd0 tx=0x7fe0f4007b00 comp rx=0 tx=0).stop 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 msgr2=0x7fe0fc19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00057c0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 shutdown_connections 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fe0e40b2880 0x7fe0e40b4d40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc10fcb0 0x7fe0fc1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe0fc075970 0x7fe0fc19bda0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 --2- 172.21.15.90:0/3602974661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0fc076290 0x7fe0fc19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:02.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 >> 172.21.15.90:0/3602974661 conn(0x7fe0fc0fefc0 msgr2=0x7fe0fc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:02.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 shutdown_connections 2024-06-02T15:54:02.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:02.501+0000 7fe102000700 1 -- 172.21.15.90:0/3602974661 wait complete. 2024-06-02T15:54:03.360 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:54:03.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:54:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:03 smithi120 ceph-mon[36451]: pgmap v802: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 532 KiB/s wr, 136 op/s 2024-06-02T15:54:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:03 smithi135 ceph-mon[36839]: pgmap v802: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 532 KiB/s wr, 136 op/s 2024-06-02T15:54:03.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:03 smithi090 ceph-mon[31595]: pgmap v802: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 532 KiB/s wr, 136 op/s 2024-06-02T15:54:03.801 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:04.511 INFO:tasks.workunit.client.0.smithi090.stdout:ok 2024-06-02T15:54:04.520 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:54:04.520 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:54:04.616 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2024-06-02T15:54:04.617 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2024-06-02T15:54:05.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:05 smithi090 ceph-mon[31595]: pgmap v803: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 514 KiB/s wr, 111 op/s 2024-06-02T15:54:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:05 smithi120 ceph-mon[36451]: pgmap v803: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 514 KiB/s wr, 111 op/s 2024-06-02T15:54:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:05 smithi135 ceph-mon[36839]: pgmap v803: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 514 KiB/s wr, 111 op/s 2024-06-02T15:54:06.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 -- 172.21.15.90:0/2581878971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 msgr2=0x7f5570109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:06.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 --2- 172.21.15.90:0/2581878971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f5570109f00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f555c009990 tx=0x7f555c009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 -- 172.21.15.90:0/2581878971 shutdown_connections 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 --2- 172.21.15.90:0/2581878971 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010a440 0x7f55701168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 --2- 172.21.15.90:0/2581878971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010e950 0x7f557010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 --2- 172.21.15.90:0/2581878971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f5570109f00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 -- 172.21.15.90:0/2581878971 >> 172.21.15.90:0/2581878971 conn(0x7f55700792d0 msgr2=0x7f5570108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 -- 172.21.15.90:0/2581878971 shutdown_connections 2024-06-02T15:54:06.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.163+0000 7f5575600700 1 -- 172.21.15.90:0/2581878971 wait complete. 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.164+0000 7f5575600700 1 Processor -- start 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 -- start start 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f55701a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010a440 0x7f55701a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 0x7f55701a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119c80 con 0x7f5570109a80 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119b00 con 0x7f557010e950 2024-06-02T15:54:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f5575600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5570119e00 con 0x7f557010a440 2024-06-02T15:54:06.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.165+0000 7f556fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f55701a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:06.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f556f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010a440 0x7f55701a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:06.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f556fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f55701a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50352/0 (socket says 172.21.15.90:50352) 2024-06-02T15:54:06.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f556fe00700 1 -- 172.21.15.90:0/3042889885 learned_addr learned my addr 172.21.15.90:0/3042889885 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 0x7f55701a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 -- 172.21.15.90:0/3042889885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010a440 msgr2=0x7f55701a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010a440 0x7f55701a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 -- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 msgr2=0x7f55701a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f55701a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 -- 172.21.15.90:0/3042889885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701a51a0 con 0x7f557010e950 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.166+0000 7f5574c00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 0x7f55701a4a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f556800efc0 tx=0x7f556800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.167+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55680092e0 con 0x7f557010e950 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.167+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5568004bb0 con 0x7f557010e950 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.167+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568004020 con 0x7f557010e950 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.167+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55701a5430 con 0x7f557010e950 2024-06-02T15:54:06.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.167+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701acd10 con 0x7f557010e950 2024-06-02T15:54:06.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.168+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557006a230 con 0x7f557010e950 2024-06-02T15:54:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.170+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f556801b030 con 0x7f557010e950 2024-06-02T15:54:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.172+0000 7f556cc00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 0x7f55540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.172+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f55680e2a20 con 0x7f557010e950 2024-06-02T15:54:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.172+0000 7f556fe00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 0x7f55540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.173+0000 7f556fe00700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 0x7f55540b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ce0 tx=0x7f5560009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.174+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5568022020 con 0x7f557010e950 2024-06-02T15:54:06.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.461+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f557002e560 con 0x7f557010e950 2024-06-02T15:54:06.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.462+0000 7f556cc00700 1 -- 172.21.15.90:0/3042889885 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11887 (secure 0 0 0) 0x7f5568010040 con 0x7f557010e950 2024-06-02T15:54:06.459 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:54:06.460 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:51:09.482439+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:54:06.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 msgr2=0x7f55540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:06.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 0x7f55540b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ce0 tx=0x7f5560009260 comp rx=0 tx=0).stop 2024-06-02T15:54:06.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 msgr2=0x7f55701a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 0x7f55701a4a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f556800efc0 tx=0x7f556800c3a0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 shutdown_connections 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f55540b2900 0x7f55540b4dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f557010e950 0x7f55701a4a60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5570109a80 0x7f55701a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 --2- 172.21.15.90:0/3042889885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f557010a440 0x7f55701a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 >> 172.21.15.90:0/3042889885 conn(0x7f55700792d0 msgr2=0x7f5570079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 shutdown_connections 2024-06-02T15:54:06.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:06.465+0000 7f5575600700 1 -- 172.21.15.90:0/3042889885 wait complete. 2024-06-02T15:54:06.464 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-06-02T15:54:07.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:54:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:07 smithi120 ceph-mon[36451]: pgmap v804: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 69 op/s 2024-06-02T15:54:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:07 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3042889885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:07 smithi135 ceph-mon[36839]: pgmap v804: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 69 op/s 2024-06-02T15:54:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:07 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3042889885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:07 smithi090 ceph-mon[31595]: pgmap v804: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 185 KiB/s wr, 69 op/s 2024-06-02T15:54:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:07 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3042889885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:09.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:09 smithi090 ceph-mon[31595]: pgmap v805: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-06-02T15:54:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:09 smithi120 ceph-mon[36451]: pgmap v805: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-06-02T15:54:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:09 smithi135 ceph-mon[36839]: pgmap v805: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.7 KiB/s wr, 23 op/s 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 -- 172.21.15.90:0/2756183007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 msgr2=0x7fda9c1065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 --2- 172.21.15.90:0/2756183007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1065f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009900 tx=0x7fda8c02a3f0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 -- 172.21.15.90:0/2756183007 shutdown_connections 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 --2- 172.21.15.90:0/2756183007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c1050c0 0x7fda9c10cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 --2- 172.21.15.90:0/2756183007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c104700 0x7fda9c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 --2- 172.21.15.90:0/2756183007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1065f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 -- 172.21.15.90:0/2756183007 >> 172.21.15.90:0/2756183007 conn(0x7fda9c100060 msgr2=0x7fda9c1024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 -- 172.21.15.90:0/2756183007 shutdown_connections 2024-06-02T15:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.615+0000 7fda9b200640 1 -- 172.21.15.90:0/2756183007 wait complete. 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 Processor -- start 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 -- start start 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c104700 0x7fda9c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c1050c0 0x7fda9c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c110760 con 0x7fda9c1061d0 2024-06-02T15:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1105e0 con 0x7fda9c104700 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.616+0000 7fda9b200640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1108e0 con 0x7fda9c1050c0 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50370/0 (socket says 172.21.15.90:50370) 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 -- 172.21.15.90:0/1811303147 learned_addr learned my addr 172.21.15.90:0/1811303147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda99e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c104700 0x7fda9c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda99400640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c1050c0 0x7fda9c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 -- 172.21.15.90:0/1811303147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c1050c0 msgr2=0x7fda9c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c1050c0 0x7fda9c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 -- 172.21.15.90:0/1811303147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c104700 msgr2=0x7fda9c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c104700 0x7fda9c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.617+0000 7fda9a800640 1 -- 172.21.15.90:0/1811303147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1a2a30 con 0x7fda9c1061d0 2024-06-02T15:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda9a800640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fda9400eb10 tx=0x7fda9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda92a00640 1 -- 172.21.15.90:0/1811303147 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94004b20 con 0x7fda9c1061d0 2024-06-02T15:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda92a00640 1 -- 172.21.15.90:0/1811303147 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda94004cc0 con 0x7fda9c1061d0 2024-06-02T15:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a2d20 con 0x7fda9c1061d0 2024-06-02T15:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda92a00640 1 -- 172.21.15.90:0/1811303147 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94010670 con 0x7fda9c1061d0 2024-06-02T15:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.618+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a3210 con 0x7fda9c1061d0 2024-06-02T15:54:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.621+0000 7fdaa3834740 1 Processor -- start 2024-06-02T15:54:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.621+0000 7fda92a00640 1 -- 172.21.15.90:0/1811303147 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fda94005020 con 0x7fda9c1061d0 2024-06-02T15:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 -- start start 2024-06-02T15:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 0x556eb9024110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb90291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9029850 0x556eb902bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556eb8f93b90 con 0x556eb9029850 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556eb8f83730 con 0x556eb9023cf0 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fdaa3834740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556eb8f87770 con 0x556eb90246e0 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fda99400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 0x556eb9024110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fda99e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb90291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fda99400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 0x556eb9024110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:55288/0 (socket says 172.21.15.90:55288) 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fda99400640 1 -- 172.21.15.90:0/3841604011 learned_addr learned my addr 172.21.15.90:0/3841604011 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.622+0000 7fda99400640 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9029850 0x556eb902bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 -- 172.21.15.90:0/3841604011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 msgr2=0x556eb9024110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 0x556eb9024110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 -- 172.21.15.90:0/3841604011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9029850 msgr2=0x556eb902bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9029850 0x556eb902bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 -- 172.21.15.90:0/3841604011 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb8f1c8b0 con 0x556eb90246e0 2024-06-02T15:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda92a00640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 0x7fda800b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99e00640 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb90291e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02a910 tx=0x7fda8c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bb0c1681424d0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda92a00640 1 -- 172.21.15.90:0/1811303147 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fda940e2ac0 con 0x7fda9c1061d0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda99400640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 0x7fda800b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.623+0000 7fda87000640 1 -- 172.21.15.90:0/3841604011 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c035030 con 0x556eb90246e0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fda87000640 1 -- 172.21.15.90:0/3841604011 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c040070 con 0x556eb90246e0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fda87000640 1 -- 172.21.15.90:0/3841604011 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c031800 con 0x556eb90246e0 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 -- 172.21.15.90:0/3841604011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 msgr2=0x556eb90291e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fda99400640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 0x7fda800b4d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x556eb9024f40 tx=0x7fda88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb90291e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02a910 tx=0x7fda8c004040 comp rx=0 tx=0).stop 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 -- 172.21.15.90:0/3841604011 shutdown_connections 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9029850 0x556eb902bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb90291e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 --2- 172.21.15.90:0/3841604011 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9023cf0 0x556eb9024110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.624+0000 7fdaa3834740 1 -- 172.21.15.90:0/3841604011 >> 172.21.15.90:0/3841604011 conn(0x556eb901f360 msgr2=0x556eb901f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.625+0000 7fdaa3834740 1 -- 172.21.15.90:0/3841604011 shutdown_connections 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.625+0000 7fdaa3834740 1 -- 172.21.15.90:0/3841604011 wait complete. 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.625+0000 7fdaa3834740 1 -- start start 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.625+0000 7fdaa3834740 1 Processor -- start 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9034fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556eb8c69790 con 0x556eb9023cf0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556eb8f2bbc0 con 0x556eb9029850 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556eb902c7a0 con 0x556eb90246e0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fdaa3834740 10 client.0 _ll_register_callbacks cb 0x556eb8f09400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50390/0 (socket says 172.21.15.90:50390) 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 -- 172.21.15.90:0/307749735 learned_addr learned my addr 172.21.15.90:0/307749735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99400640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda9a800640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9034fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 -- 172.21.15.90:0/307749735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 msgr2=0x556eb902df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 -- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 msgr2=0x556eb9034fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9034fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda99e00640 1 -- 172.21.15.90:0/307749735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb8c98210 con 0x556eb9023cf0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.626+0000 7fda9a800640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9034fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fda99e00640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda8c031500 tx=0x7fda8c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9dcd9983d4b59b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fda7b400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c035070 con 0x556eb9023cf0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fda8c0315a0 con 0x556eb9023cf0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda8c040070 con 0x556eb9023cf0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fda8c009520 con 0x556eb9023cf0 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fdaa3834740 10 client.16146 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.627+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556eb8d64620 con 0x556eb9023cf0 2024-06-02T15:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c040070 con 0x556eb9023cf0 2024-06-02T15:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fda8c04d030 con 0x556eb9023cf0 2024-06-02T15:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 10 client.16146 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 20 client.16146 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb9039130 0x556eb903b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 4 client.16146 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eb8f95ae0 con 0x556eb9039130 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.628+0000 7fda99400640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb9039130 0x556eb903b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.629+0000 7fda99400640 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb9039130 0x556eb903b520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9270eaa81cf4e7d server_cookie=49e3bf006693ca09 in_seq=0 out_seq=0 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.629+0000 7fda7b400640 10 client.16146 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fda7b400640 1 -- 172.21.15.90:0/307749735 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556eb8f95ae0 con 0x556eb9039130 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fda7b400640 10 client.16146 handle_command_reply: tid=0 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 client.16146 shutdown 2024-06-02T15:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 msgr2=0x556eb902da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fda8c031500 tx=0x7fda8c004460 comp rx=0 tx=0).stop 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 shutdown_connections 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb9039130 0x556eb903b520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9034fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 --2- 172.21.15.90:0/307749735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902da50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 >> 172.21.15.90:0/307749735 conn(0x556eb901f360 msgr2=0x556eb901e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fda7b400640 0 client.16146 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 shutdown_connections 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 1 -- 172.21.15.90:0/307749735 wait complete. 2024-06-02T15:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.630+0000 7fdaa3834740 20 client.16146 trim_cache size 0 max 16384 2024-06-02T15:54:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.649+0000 7fdaa3834740 1 Processor -- start 2024-06-02T15:54:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.649+0000 7fdaa3834740 1 -- start start 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb902daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1affd0 con 0x556eb9023cf0 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ad7e0 con 0x556eb9029850 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fdaa3834740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ab630 con 0x556eb90246e0 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fda99e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fda9a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb902daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fda9a800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb902daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:55296/0 (socket says 172.21.15.90:55296) 2024-06-02T15:54:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.650+0000 7fda9a800640 1 -- 172.21.15.90:0/341236123 learned_addr learned my addr 172.21.15.90:0/341236123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99400640 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 -- 172.21.15.90:0/341236123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 msgr2=0x556eb902d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 -- 172.21.15.90:0/341236123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 msgr2=0x556eb902daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb902daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 -- 172.21.15.90:0/341236123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eb902c8e0 con 0x556eb9023cf0 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda99e00640 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902d070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02abe0 tx=0x7fda8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77f906931c9fdf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.651+0000 7fda86600640 1 -- 172.21.15.90:0/341236123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c035040 con 0x556eb9023cf0 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fda86600640 1 -- 172.21.15.90:0/341236123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c05c070 con 0x556eb9023cf0 2024-06-02T15:54:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fda86600640 1 -- 172.21.15.90:0/341236123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c031b70 con 0x556eb9023cf0 2024-06-02T15:54:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 -- 172.21.15.90:0/341236123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 msgr2=0x556eb902d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902d070 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fda8c02abe0 tx=0x7fda8c000bc0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 -- 172.21.15.90:0/341236123 shutdown_connections 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb902daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb90246e0 0x556eb902d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.652+0000 7fdaa3834740 1 --2- 172.21.15.90:0/341236123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb9023cf0 0x556eb902d070 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.653+0000 7fdaa3834740 1 -- 172.21.15.90:0/341236123 >> 172.21.15.90:0/341236123 conn(0x556eb901f360 msgr2=0x556eb90259e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.653+0000 7fdaa3834740 1 -- 172.21.15.90:0/341236123 shutdown_connections 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.653+0000 7fdaa3834740 1 -- 172.21.15.90:0/341236123 wait complete. 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.653+0000 7fdaa3834740 1 -- start start 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.653+0000 7fdaa3834740 1 Processor -- start 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 0x556eb90319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 0x556eb9031f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9032f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1abca0 con 0x556eb90246e0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ad110 con 0x556eb9029850 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1ac370 con 0x556eb9023cf0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fdaa3834740 10 client.0 _ll_register_callbacks cb 0x7fda88009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 0x556eb9031f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 0x556eb90319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 0x556eb9031f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50418/0 (socket says 172.21.15.90:50418) 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99400640 1 -- 172.21.15.90:0/2919172791 learned_addr learned my addr 172.21.15.90:0/2919172791 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda9a800640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9032f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 -- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 msgr2=0x556eb9032f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9032f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 -- 172.21.15.90:0/2919172791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 msgr2=0x556eb9031f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 0x556eb9031f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.654+0000 7fda99e00640 1 -- 172.21.15.90:0/2919172791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eb9033520 con 0x556eb9023cf0 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fda99e00640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 0x556eb90319e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0456d0 tx=0x7fda8c045b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51f88453ddc5fb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fda7b400640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c065060 con 0x556eb9023cf0 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda8c05c070 con 0x556eb9023cf0 2024-06-02T15:54:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fda8c003cd0 con 0x556eb9023cf0 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fda8c06c050 con 0x556eb9023cf0 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fdaa3834740 10 client.26386 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.655+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556eb9034830 con 0x556eb9023cf0 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c05c070 con 0x556eb9023cf0 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fda8c04d040 con 0x556eb9023cf0 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 10 client.26386 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 10 client.26386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 20 client.26386 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb90a4150 0x556eb90a6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 4 client.26386 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eb8f8f2c0 con 0x556eb90a4150 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.656+0000 7fda99400640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb90a4150 0x556eb90a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.657+0000 7fda99400640 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb90a4150 0x556eb90a6570 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3ac213704c71b8 server_cookie=1900edc233ef9d06 in_seq=0 out_seq=0 2024-06-02T15:54:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.657+0000 7fda7b400640 10 client.26386 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.674+0000 7fda7b400640 1 -- 172.21.15.90:0/2919172791 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556eb8f8f2c0 con 0x556eb90a4150 2024-06-02T15:54:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fda7b400640 10 client.26386 handle_command_reply: tid=0 2024-06-02T15:54:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 client.26386 shutdown 2024-06-02T15:54:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 msgr2=0x556eb90319e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 0x556eb90319e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda8c0456d0 tx=0x7fda8c045b00 comp rx=0 tx=0).stop 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 shutdown_connections 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556eb90a4150 0x556eb90a6570 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556eb9029850 0x556eb9032f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556eb90246e0 0x556eb9031f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 --2- 172.21.15.90:0/2919172791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556eb9023cf0 0x556eb90319e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 >> 172.21.15.90:0/2919172791 conn(0x556eb901f360 msgr2=0x556eb901dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fda7b400640 0 client.26386 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 shutdown_connections 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.675+0000 7fdaa3834740 1 -- 172.21.15.90:0/2919172791 wait complete. 2024-06-02T15:54:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fdaa3834740 20 client.26386 trim_cache size 0 max 16384 2024-06-02T15:54:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-06-02T15:54:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-06-02T15:54:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-06-02T15:54:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 msgr2=0x7fda800b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 0x7fda800b4d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x556eb9024f40 tx=0x7fda88005f90 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 msgr2=0x7fda9c1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fda9400eb10 tx=0x7fda9400c2d0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 shutdown_connections 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fda800b2860 0x7fda800b4d20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c1061d0 0x7fda9c1a23c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda9c1050c0 0x7fda9c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 --2- 172.21.15.90:0/1811303147 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fda9c104700 0x7fda9c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.676+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 >> 172.21.15.90:0/1811303147 conn(0x7fda9c100060 msgr2=0x7fda9c1090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.677+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 shutdown_connections 2024-06-02T15:54:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:09.677+0000 7fda9b200640 1 -- 172.21.15.90:0/1811303147 wait complete. 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 -- 172.21.15.90:0/826714578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 msgr2=0x7f10b810ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 --2- 172.21.15.90:0/826714578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b810ff20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009f90 tx=0x7f10b400b330 comp rx=0 tx=0).stop 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 -- 172.21.15.90:0/826714578 shutdown_connections 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 --2- 172.21.15.90:0/826714578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b810ff20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 --2- 172.21.15.90:0/826714578 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104630 0x7f10b8104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 --2- 172.21.15.90:0/826714578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104f50 0x7f10b810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 -- 172.21.15.90:0/826714578 >> 172.21.15.90:0/826714578 conn(0x7f10b80fde60 msgr2=0x7f10b8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 -- 172.21.15.90:0/826714578 shutdown_connections 2024-06-02T15:54:09.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.966+0000 7f10bf200700 1 -- 172.21.15.90:0/826714578 wait complete. 2024-06-02T15:54:09.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 Processor -- start 2024-06-02T15:54:09.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 -- start start 2024-06-02T15:54:09.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104630 0x7f10b819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104f50 0x7f10b819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8113660 con 0x7f10b810da60 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81134e0 con 0x7f10b8104f50 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.968+0000 7f10bf200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81137e0 con 0x7f10b8104630 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10bde00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104630 0x7f10b819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50436/0 (socket says 172.21.15.90:50436) 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 -- 172.21.15.90:0/3353962058 learned_addr learned my addr 172.21.15.90:0/3353962058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10bd400700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104f50 0x7f10b819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 -- 172.21.15.90:0/3353962058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104630 msgr2=0x7f10b819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104630 0x7f10b819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 -- 172.21.15.90:0/3353962058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104f50 msgr2=0x7f10b819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104f50 0x7f10b819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.969+0000 7f10be800700 1 -- 172.21.15.90:0/3353962058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b819a040 con 0x7f10b810da60 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10be800700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009790 tx=0x7f10b400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10b2a00700 1 -- 172.21.15.90:0/3353962058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4021070 con 0x7f10b810da60 2024-06-02T15:54:09.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10b2a00700 1 -- 172.21.15.90:0/3353962058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b400ce50 con 0x7f10b810da60 2024-06-02T15:54:09.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b819a2d0 con 0x7f10b810da60 2024-06-02T15:54:09.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10b2a00700 1 -- 172.21.15.90:0/3353962058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4005040 con 0x7f10b810da60 2024-06-02T15:54:09.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.970+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b81021c0 con 0x7f10b810da60 2024-06-02T15:54:09.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.974+0000 7f10b2a00700 1 -- 172.21.15.90:0/3353962058 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f10b4024070 con 0x7f10b810da60 2024-06-02T15:54:09.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 Processor -- start 2024-06-02T15:54:09.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 -- start start 2024-06-02T15:54:09.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10b2a00700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 0x7f10a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bf2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065bf37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c065b50e40 con 0x55c065bf87c0 2024-06-02T15:54:09.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c065b409e0 con 0x55c065bf90e0 2024-06-02T15:54:09.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c065b44a20 con 0x55c065bf3390 2024-06-02T15:54:09.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.975+0000 7f10b2a00700 1 -- 172.21.15.90:0/3353962058 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f10b40e3430 con 0x7f10b810da60 2024-06-02T15:54:09.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bd400700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 0x7f10a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bd400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bf2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50438/0 (socket says 172.21.15.90:50438) 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 -- 172.21.15.90:0/1051774573 learned_addr learned my addr 172.21.15.90:0/1051774573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10be800700 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065bf37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 -- 172.21.15.90:0/1051774573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 msgr2=0x55c065bf37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065bf37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 -- 172.21.15.90:0/1051774573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 msgr2=0x55c065bf2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bf2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.976+0000 7f10bde00700 1 -- 172.21.15.90:0/1051774573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c065b09420 con 0x55c065bf87c0 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.977+0000 7f10bd400700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 0x7f10a00b4dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10b810ee10 tx=0x7f10a8009ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.977+0000 7f10bde00700 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ea0 tx=0x7f10ac011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9949234abe67da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.977+0000 7f10a7400700 1 -- 172.21.15.90:0/1051774573 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004310 con 0x55c065bf87c0 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.977+0000 7f10a7400700 1 -- 172.21.15.90:0/1051774573 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac0044b0 con 0x55c065bf87c0 2024-06-02T15:54:09.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.977+0000 7f10a7400700 1 -- 172.21.15.90:0/1051774573 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac019870 con 0x55c065bf87c0 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1051774573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 msgr2=0x55c065bf8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009ea0 tx=0x7f10ac011720 comp rx=0 tx=0).stop 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1051774573 shutdown_connections 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065bf37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065bf8ba0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1051774573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bf2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1051774573 >> 172.21.15.90:0/1051774573 conn(0x55c065b20be0 msgr2=0x55c065bed810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1051774573 shutdown_connections 2024-06-02T15:54:09.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.978+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1051774573 wait complete. 2024-06-02T15:54:09.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.979+0000 7f10cb0c1b80 1 -- start start 2024-06-02T15:54:09.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.979+0000 7f10cb0c1b80 1 Processor -- start 2024-06-02T15:54:09.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bfed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 0x55c065bfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c065a46fa0 con 0x55c065bf3390 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55c065ae1760 con 0x55c065bf87c0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c065bffc00 con 0x55c065bf90e0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10cb0c1b80 10 client.0 _ll_register_callbacks cb 0x55c065aeac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10bd400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10be800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 0x55c065bfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10bd400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:55350/0 (socket says 172.21.15.90:55350) 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10bd400700 1 -- 172.21.15.90:0/3285078728 learned_addr learned my addr 172.21.15.90:0/3285078728 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.980+0000 7f10bde00700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bfed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 -- 172.21.15.90:0/3285078728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 msgr2=0x55c065bfc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 0x55c065bfc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 -- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 msgr2=0x55c065bfed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bfed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 -- 172.21.15.90:0/3285078728 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c065a908d0 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10be800700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 0x55c065bfc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bde00700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bfed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10bd400700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f10a8004450 tx=0x7f10a80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271b2611b254a0f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f109be00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004480 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac0044b0 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.981+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f10a8003680 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f10a8003880 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 10 client.26462 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c065b11060 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac0044b0 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f10a8020050 con 0x55c065bf87c0 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 10 client.26462 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:09.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 10 client.26462 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:54:09.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 20 client.26462 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:09.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c0c740 0x55c065c0ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 4 client.26462 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:09.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.982+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c065b52d90 con 0x55c065c0c740 2024-06-02T15:54:09.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.983+0000 7f10bde00700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c0c740 0x55c065c0ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.983+0000 7f109be00700 10 client.26462 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.983+0000 7f10bde00700 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c0c740 0x55c065c0ebe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3615a827b74fce57 server_cookie=2959da96eed5225d in_seq=0 out_seq=0 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f109be00700 1 -- 172.21.15.90:0/3285078728 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c065b52d90 con 0x55c065c0c740 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f109be00700 10 client.26462 handle_command_reply: tid=0 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f10cb0c1b80 1 client.26462 shutdown 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 msgr2=0x55c065bfc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f10a8004450 tx=0x7f10a80047c0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 shutdown_connections 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.984+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c0c740 0x55c065c0ebe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf90e0 0x55c065bfc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bfed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3285078728 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf87c0 0x55c065bfc100 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 >> 172.21.15.90:0/3285078728 conn(0x55c065b20be0 msgr2=0x55c065becfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f109be00700 0 client.26462 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 shutdown_connections 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3285078728 wait complete. 2024-06-02T15:54:09.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:09.985+0000 7f10cb0c1b80 20 client.26462 trim_cache size 0 max 16384 2024-06-02T15:54:09.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.001+0000 7f10cb0c1b80 1 Processor -- start 2024-06-02T15:54:09.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.001+0000 7f10cb0c1b80 1 -- start start 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 0x55c065bef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bed740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b3ce0 con 0x55c065bf3390 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b14b0 con 0x55c065bf90e0 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81af320 con 0x55c065bf87c0 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10bde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50456/0 (socket says 172.21.15.90:50456) 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10bde00700 1 -- 172.21.15.90:0/1372727503 learned_addr learned my addr 172.21.15.90:0/1372727503 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10bd400700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 0x55c065bef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:09.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.002+0000 7f10be800700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bed740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 -- 172.21.15.90:0/1372727503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 msgr2=0x55c065bef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 0x55c065bef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 -- 172.21.15.90:0/1372727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 msgr2=0x55c065bed740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bed740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 -- 172.21.15.90:0/1372727503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c065bf8ee0 con 0x55c065bf3390 2024-06-02T15:54:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bd400700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 0x55c065bef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10bde00700 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f10ac014040 tx=0x7f10ac005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ca9d9b473b3920 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10a6a00700 1 -- 172.21.15.90:0/1372727503 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac031070 con 0x55c065bf3390 2024-06-02T15:54:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10a6a00700 1 -- 172.21.15.90:0/1372727503 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac005500 con 0x55c065bf3390 2024-06-02T15:54:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.003+0000 7f10a6a00700 1 -- 172.21.15.90:0/1372727503 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004ae0 con 0x55c065bf3390 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1372727503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 msgr2=0x55c065bf2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f10ac014040 tx=0x7f10ac005a90 comp rx=0 tx=0).stop 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1372727503 shutdown_connections 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065bed740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf3390 0x55c065bf2d70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/1372727503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf87c0 0x55c065bef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1372727503 >> 172.21.15.90:0/1372727503 conn(0x55c065b20be0 msgr2=0x55c065bf6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1372727503 shutdown_connections 2024-06-02T15:54:10.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.004+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/1372727503 wait complete. 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 -- start start 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 Processor -- start 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065c14d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065c05600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81af970 con 0x55c065bf87c0 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b0de0 con 0x55c065bf90e0 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81b0040 con 0x55c065bf3390 2024-06-02T15:54:10.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.005+0000 7f10cb0c1b80 10 client.0 _ll_register_callbacks cb 0x55c065aeac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:10.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:10.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bde00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065c14d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:10.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50462/0 (socket says 172.21.15.90:50462) 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 -- 172.21.15.90:0/3605813600 learned_addr learned my addr 172.21.15.90:0/3605813600 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10be800700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065c05600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 -- 172.21.15.90:0/3605813600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 msgr2=0x55c065c14d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065c14d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 -- 172.21.15.90:0/3605813600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 msgr2=0x55c065c05600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065c05600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 -- 172.21.15.90:0/3605813600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c065c13610 con 0x55c065bf87c0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bde00700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065c14d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f10bd400700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55c065bef3c0 tx=0x7f10a8013580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bd8a048e6cee1a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.006+0000 7f109be00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8003680 con 0x55c065bf87c0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f10a8003a20 con 0x55c065bf87c0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ac005500 con 0x55c065bf87c0 2024-06-02T15:54:10.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f10a8045050 con 0x55c065bf87c0 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f10cb0c1b80 10 client.16170 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.007+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c065c767d0 con 0x55c065bf87c0 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac005500 con 0x55c065bf87c0 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f10a8040030 con 0x55c065bf87c0 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 10 client.16170 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 20 client.16170 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c76f90 0x55c065c793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 4 client.16170 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:10.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c065b4c570 con 0x55c065c76f90 2024-06-02T15:54:10.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.008+0000 7f10bde00700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c76f90 0x55c065c793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:10.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10bde00700 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c76f90 0x55c065c793b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d24339ef2e2af19 server_cookie=def2516e71bcd98 in_seq=0 out_seq=0 2024-06-02T15:54:10.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f109be00700 10 client.16170 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f109be00700 1 -- 172.21.15.90:0/3605813600 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c065b4c570 con 0x55c065c76f90 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f109be00700 10 client.16170 handle_command_reply: tid=0 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 client.16170 shutdown 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 msgr2=0x55c065c04ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55c065bef3c0 tx=0x7f10a8013580 comp rx=0 tx=0).stop 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 shutdown_connections 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55c065c76f90 0x55c065c793b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55c065bf90e0 0x55c065c05600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c065bf3390 0x55c065c14d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 --2- 172.21.15.90:0/3605813600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c065bf87c0 0x55c065c04ec0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 >> 172.21.15.90:0/3605813600 conn(0x55c065b20be0 msgr2=0x55c065bfa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.009+0000 7f109be00700 0 client.16170 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.010+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 shutdown_connections 2024-06-02T15:54:10.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.010+0000 7f10cb0c1b80 1 -- 172.21.15.90:0/3605813600 wait complete. 2024-06-02T15:54:10.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.010+0000 7f10cb0c1b80 20 client.16170 trim_cache size 0 max 16384 2024-06-02T15:54:10.009 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:10.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 msgr2=0x7f10a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:10.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 0x7f10a00b4dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10b810ee10 tx=0x7f10a8009ab0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 msgr2=0x7f10b81998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009790 tx=0x7f10b400cc30 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 shutdown_connections 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f10a00b2900 0x7f10a00b4dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10b810da60 0x7f10b81998e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10b8104630 0x7f10b819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 --2- 172.21.15.90:0/3353962058 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10b8104f50 0x7f10b819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 >> 172.21.15.90:0/3353962058 conn(0x7f10b80fde60 msgr2=0x7f10b80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 shutdown_connections 2024-06-02T15:54:10.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:10.011+0000 7f10bf200700 1 -- 172.21.15.90:0/3353962058 wait complete. 2024-06-02T15:54:11.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:54:11.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:11 smithi120 ceph-mon[36451]: pgmap v806: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-06-02T15:54:11.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:11 smithi135 ceph-mon[36839]: pgmap v806: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-06-02T15:54:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:11 smithi090 ceph-mon[31595]: pgmap v806: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-06-02T15:54:11.665 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:13 smithi120 ceph-mon[36451]: pgmap v807: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 21 op/s 2024-06-02T15:54:13.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:13 smithi090 ceph-mon[31595]: pgmap v807: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 21 op/s 2024-06-02T15:54:13.656 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:13 smithi135 ceph-mon[36839]: pgmap v807: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 KiB/s wr, 21 op/s 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 -- 172.21.15.90:0/733074695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4105760 msgr2=0x7f26b4105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 --2- 172.21.15.90:0/733074695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4105760 0x7f26b4105be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098f0 tx=0x7f26a0009c00 comp rx=0 tx=0).stop 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 -- 172.21.15.90:0/733074695 shutdown_connections 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 --2- 172.21.15.90:0/733074695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b4106120 0x7f26b4112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 --2- 172.21.15.90:0/733074695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b410b760 0x7f26b410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.185+0000 7f26bba00700 1 --2- 172.21.15.90:0/733074695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4105760 0x7f26b4105be0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.186+0000 7f26bba00700 1 -- 172.21.15.90:0/733074695 >> 172.21.15.90:0/733074695 conn(0x7f26b4101210 msgr2=0x7f26b4103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.186+0000 7f26bba00700 1 -- 172.21.15.90:0/733074695 shutdown_connections 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.186+0000 7f26bba00700 1 -- 172.21.15.90:0/733074695 wait complete. 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 Processor -- start 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 -- start start 2024-06-02T15:54:14.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4106120 0x7f26b419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b410b760 0x7f26b41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41159f0 con 0x7f26b410b760 2024-06-02T15:54:14.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4115870 con 0x7f26b4105760 2024-06-02T15:54:14.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.187+0000 7f26bba00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4115b70 con 0x7f26b4106120 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b410b760 0x7f26b41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35468/0 (socket says 172.21.15.90:35468) 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 -- 172.21.15.90:0/1316788053 learned_addr learned my addr 172.21.15.90:0/1316788053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b3e00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4106120 0x7f26b419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 -- 172.21.15.90:0/1316788053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4106120 msgr2=0x7f26b419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4106120 0x7f26b419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 -- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b410b760 msgr2=0x7f26b41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b410b760 0x7f26b41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.188+0000 7f26b9000700 1 -- 172.21.15.90:0/1316788053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b41a0e60 con 0x7f26b4105760 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26b9000700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9a0 tx=0x7f26a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26b1600700 1 -- 172.21.15.90:0/1316788053 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a800ca10 con 0x7f26b4105760 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26b1600700 1 -- 172.21.15.90:0/1316788053 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a800cbb0 con 0x7f26b4105760 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b41a1150 con 0x7f26b4105760 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26b1600700 1 -- 172.21.15.90:0/1316788053 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8018680 con 0x7f26b4105760 2024-06-02T15:54:14.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.189+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a89d0 con 0x7f26b4105760 2024-06-02T15:54:14.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.193+0000 7f26b1600700 1 -- 172.21.15.90:0/1316788053 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26a8010430 con 0x7f26b4105760 2024-06-02T15:54:14.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.194+0000 7f26c641bb80 1 Processor -- start 2024-06-02T15:54:14.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.194+0000 7f26c641bb80 1 -- start start 2024-06-02T15:54:14.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.194+0000 7f26b1600700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 0x7f269c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b1600700 1 -- 172.21.15.90:0/1316788053 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f26a80e26a0 con 0x7f26b4105760 2024-06-02T15:54:14.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f18c500e40 con 0x55f18c5a8f30 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f18c4f09e0 con 0x55f18c5a8610 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26c641bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f18c4f4a20 con 0x55f18c5a3140 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b9a00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 0x7f269c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b9000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55146/0 (socket says 172.21.15.90:55146) 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b3e00700 1 -- 172.21.15.90:0/1471420191 learned_addr learned my addr 172.21.15.90:0/1471420191 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b9000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5a89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35474/0 (socket says 172.21.15.90:35474) 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.195+0000 7f26b9a00700 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 -- 172.21.15.90:0/1471420191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 msgr2=0x55f18c5a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 -- 172.21.15.90:0/1471420191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 msgr2=0x55f18c5a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 -- 172.21.15.90:0/1471420191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c4b9420 con 0x55f18c5a8f30 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b9a00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 0x7f269c0b4dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00f5a0 tx=0x7f26ac00f960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26b3e00700 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098f0 tx=0x7f26a000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2965a18ef93488b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26a6000700 1 -- 172.21.15.90:0/1471420191 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0026070 con 0x55f18c5a8f30 2024-06-02T15:54:14.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.196+0000 7f26a6000700 1 -- 172.21.15.90:0/1471420191 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000f910 con 0x55f18c5a8f30 2024-06-02T15:54:14.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26a6000700 1 -- 172.21.15.90:0/1471420191 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0003680 con 0x55f18c5a8f30 2024-06-02T15:54:14.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 -- 172.21.15.90:0/1471420191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 msgr2=0x55f18c5a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098f0 tx=0x7f26a000bf60 comp rx=0 tx=0).stop 2024-06-02T15:54:14.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 -- 172.21.15.90:0/1471420191 shutdown_connections 2024-06-02T15:54:14.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 --2- 172.21.15.90:0/1471420191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5a2b20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.197+0000 7f26c641bb80 1 -- 172.21.15.90:0/1471420191 >> 172.21.15.90:0/1471420191 conn(0x55f18c4d0be0 msgr2=0x55f18c59d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.198+0000 7f26c641bb80 1 -- 172.21.15.90:0/1471420191 shutdown_connections 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.198+0000 7f26c641bb80 1 -- 172.21.15.90:0/1471420191 wait complete. 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.198+0000 7f26c641bb80 1 -- start start 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.198+0000 7f26c641bb80 1 Processor -- start 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f18c3f6fa0 con 0x55f18c5a8f30 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f18c491760 con 0x55f18c5a8610 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f18c5af980 con 0x55f18c5a3140 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26c641bb80 10 client.0 _ll_register_callbacks cb 0x55f18c49ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26b9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26b9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55160/0 (socket says 172.21.15.90:55160) 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26b9a00700 1 -- 172.21.15.90:0/3807091358 learned_addr learned my addr 172.21.15.90:0/3807091358 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.199+0000 7f26b9000700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 -- 172.21.15.90:0/3807091358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 msgr2=0x55f18c5aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 -- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 msgr2=0x55f18c5ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5ac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 -- 172.21.15.90:0/3807091358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c4408d0 con 0x55f18c5a8610 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b9000700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26b3e00700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5abf80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009ff0 tx=0x7f26a00058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7743e5623bd588 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f269aa00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0026040 con 0x55f18c5a8610 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f26a000f460 con 0x55f18c5a8610 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.200+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a000f910 con 0x55f18c5a8610 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f26a000f660 con 0x55f18c5a8610 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f26c641bb80 10 client.26480 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:14.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f18c4c1060 con 0x55f18c5a8610 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a000f910 con 0x55f18c5a8610 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f26a0029070 con 0x55f18c5a8610 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f26c641bb80 10 client.26480 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f26c641bb80 10 client.26480 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.201+0000 7f26c641bb80 20 client.26480 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c5bc2f0 0x55f18c5be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f26c641bb80 4 client.26480 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18c502d90 con 0x55f18c5bc2f0 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f26b9000700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c5bc2f0 0x55f18c5be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f26b9000700 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c5bc2f0 0x55f18c5be6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6a5547f660a73e5 server_cookie=1640138bb886a2f2 in_seq=0 out_seq=0 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.202+0000 7f269aa00700 10 client.26480 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.203+0000 7f269aa00700 1 -- 172.21.15.90:0/3807091358 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f18c502d90 con 0x55f18c5bc2f0 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.203+0000 7f269aa00700 10 client.26480 handle_command_reply: tid=0 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.203+0000 7f26c641bb80 1 client.26480 shutdown 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 msgr2=0x55f18c5abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5abf80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009ff0 tx=0x7f26a00058e0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 shutdown_connections 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a3140 0x55f18c5aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c5bc2f0 0x55f18c5be6e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3807091358 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8610 0x55f18c5abf80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 >> 172.21.15.90:0/3807091358 conn(0x55f18c4d0be0 msgr2=0x55f18c59cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f269aa00700 0 client.26480 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 shutdown_connections 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 1 -- 172.21.15.90:0/3807091358 wait complete. 2024-06-02T15:54:14.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.204+0000 7f26c641bb80 20 client.26480 trim_cache size 0 max 16384 2024-06-02T15:54:14.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.220+0000 7f26c641bb80 1 Processor -- start 2024-06-02T15:54:14.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.220+0000 7f26c641bb80 1 -- start start 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 0x55f18c59d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b3f10 con 0x55f18c5a3140 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b16e0 con 0x55f18c5a8f30 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26c641bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41af530 con 0x55f18c5a8610 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26b9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26b9000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55162/0 (socket says 172.21.15.90:55162) 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26b9000700 1 -- 172.21.15.90:0/3980225142 learned_addr learned my addr 172.21.15.90:0/3980225142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26b3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.221+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 0x55f18c59d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 -- 172.21.15.90:0/3980225142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 msgr2=0x55f18c5a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 -- 172.21.15.90:0/3980225142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 msgr2=0x55f18c5a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 -- 172.21.15.90:0/3980225142 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c5b8c50 con 0x55f18c5a8f30 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9000700 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 0x55f18c59d4f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009a20 tx=0x7f26ac01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be23a2e1b2a98b8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26a5600700 1 -- 172.21.15.90:0/3980225142 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac009470 con 0x55f18c5a8f30 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26a5600700 1 -- 172.21.15.90:0/3980225142 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ac008c40 con 0x55f18c5a8f30 2024-06-02T15:54:14.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.222+0000 7f26a5600700 1 -- 172.21.15.90:0/3980225142 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac004790 con 0x55f18c5a8f30 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 -- 172.21.15.90:0/3980225142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 msgr2=0x55f18c59d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 0x55f18c59d4f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009a20 tx=0x7f26ac01c000 comp rx=0 tx=0).stop 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 -- 172.21.15.90:0/3980225142 shutdown_connections 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a8f30 0x55f18c59d4f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a3140 0x55f18c5a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3980225142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 -- 172.21.15.90:0/3980225142 >> 172.21.15.90:0/3980225142 conn(0x55f18c4d0be0 msgr2=0x55f18c5a93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 -- 172.21.15.90:0/3980225142 shutdown_connections 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.223+0000 7f26c641bb80 1 -- 172.21.15.90:0/3980225142 wait complete. 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 -- start start 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 Processor -- start 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 0x55f18c5c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41afba0 con 0x55f18c5a8f30 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b1010 con 0x55f18c5a3140 2024-06-02T15:54:14.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41b0270 con 0x55f18c5a8610 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.224+0000 7f26c641bb80 10 client.0 _ll_register_callbacks cb 0x55f18c49ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55164/0 (socket says 172.21.15.90:55164) 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 -- 172.21.15.90:0/3736614525 learned_addr learned my addr 172.21.15.90:0/3736614525 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b3e00700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9000700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 0x55f18c5c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 -- 172.21.15.90:0/3736614525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 msgr2=0x55f18c5bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 -- 172.21.15.90:0/3736614525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 msgr2=0x55f18c5c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 0x55f18c5c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 -- 172.21.15.90:0/3736614525 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c5bf8f0 con 0x55f18c5a8f30 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9000700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 0x55f18c5c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:14.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.225+0000 7f26b9a00700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0048b0 tx=0x7f26ac0048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922dea7d40466244 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f269aa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac032020 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7f26ac004a70 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac008c40 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f26ac03d050 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f26c641bb80 10 client.16188 fetch_fsmap learned FSMap version 37 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f18c626840 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.226+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ac008c40 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f26ac0052b0 con 0x55f18c5a8f30 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 10 client.16188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:54:14.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 20 client.16188 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c626f70 0x55f18c629390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 4 client.16188 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f18c4fc570 con 0x55f18c626f70 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.227+0000 7f26b9000700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c626f70 0x55f18c629390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26b9000700 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c626f70 0x55f18c629390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9add902424d0c28 server_cookie=63b27466a327c7a3 in_seq=0 out_seq=0 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f269aa00700 10 client.16188 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f269aa00700 1 -- 172.21.15.90:0/3736614525 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f18c4fc570 con 0x55f18c626f70 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f269aa00700 10 client.16188 handle_command_reply: tid=0 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 client.16188 shutdown 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 msgr2=0x55f18c5bf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0048b0 tx=0x7f26ac0048e0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 shutdown_connections 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f18c5a8f30 0x55f18c5bf3b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f18c626f70 0x55f18c629390 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f18c5a3140 0x55f18c5c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 --2- 172.21.15.90:0/3736614525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f18c5a8610 0x55f18c5bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 >> 172.21.15.90:0/3736614525 conn(0x55f18c4d0be0 msgr2=0x55f18c5ad580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.228+0000 7f269aa00700 0 client.16188 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 shutdown_connections 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26c641bb80 1 -- 172.21.15.90:0/3736614525 wait complete. 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26c641bb80 20 client.16188 trim_cache size 0 max 16384 2024-06-02T15:54:14.232 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 msgr2=0x7f269c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 0x7f269c0b4dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00f5a0 tx=0x7f26ac00f960 comp rx=0 tx=0).stop 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 msgr2=0x7f26b419bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.229+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9a0 tx=0x7f26a800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 shutdown_connections 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f269c0b2900 0x7f269c0b4dc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26b410b760 0x7f26b41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f26b4105760 0x7f26b419bea0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 --2- 172.21.15.90:0/1316788053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4106120 0x7f26b419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:14.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 >> 172.21.15.90:0/1316788053 conn(0x7f26b4101210 msgr2=0x7f26b4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:14.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 shutdown_connections 2024-06-02T15:54:14.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:14.230+0000 7f26bba00700 1 -- 172.21.15.90:0/1316788053 wait complete. 2024-06-02T15:54:14.742 INFO:tasks.workunit.client.0.smithi090.stdout:ok 2024-06-02T15:54:14.752 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:54:14.752 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:54:14.860 INFO:tasks.workunit:Running workunit fs/misc/i_complete_vs_rename.sh... 2024-06-02T15:54:14.861 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/i_complete_vs_rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/i_complete_vs_rename.sh 2024-06-02T15:54:14.974 INFO:tasks.workunit.client.0.smithi090.stdout:a 2024-06-02T15:54:14.974 INFO:tasks.workunit.client.0.smithi090.stdout:b 2024-06-02T15:54:14.974 INFO:tasks.workunit.client.0.smithi090.stdout:c 2024-06-02T15:54:14.974 INFO:tasks.workunit.client.0.smithi090.stdout:d 2024-06-02T15:54:14.989 INFO:tasks.workunit.client.0.smithi090.stderr:stat: cannot statx 'e': No such file or directory 2024-06-02T15:54:15.001 INFO:tasks.workunit.client.0.smithi090.stdout:attempting rename over existing file... 2024-06-02T15:54:15.062 INFO:tasks.workunit.client.0.smithi090.stdout:f 2024-06-02T15:54:15.062 INFO:tasks.workunit.client.0.smithi090.stdout:rename over existing file is okay 2024-06-02T15:54:15.062 INFO:tasks.workunit.client.0.smithi090.stdout:attempting rename over negative dentry... 2024-06-02T15:54:15.084 INFO:tasks.workunit.client.0.smithi090.stdout:e 2024-06-02T15:54:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:rename over negative dentry is ok 2024-06-02T15:54:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:OK 2024-06-02T15:54:15.087 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:54:15.087 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:54:15.206 INFO:tasks.workunit:Running workunit fs/misc/layout_vxattrs.sh... 2024-06-02T15:54:15.206 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh 2024-06-02T15:54:15.215 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:54:15.276 INFO:tasks.workunit.client.0.smithi090.stderr:+ datapool= 2024-06-02T15:54:15.276 INFO:tasks.workunit.client.0.smithi090.stderr:+ dir=. 2024-06-02T15:54:15.276 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.276 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo . 2024-06-02T15:54:15.276 INFO:tasks.workunit.client.0.smithi090.stdout:. 2024-06-02T15:54:15.277 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfattr -n ceph.dir.layout.pool . --only-values 2024-06-02T15:54:15.279 INFO:tasks.workunit.client.0.smithi090.stderr:.: ceph.dir.layout.pool: No such attribute 2024-06-02T15:54:15.279 INFO:tasks.workunit.client.0.smithi090.stderr:+ datapool= 2024-06-02T15:54:15.279 INFO:tasks.workunit.client.0.smithi090.stdout:./.. 2024-06-02T15:54:15.279 INFO:tasks.workunit.client.0.smithi090.stderr:+ dir=./.. 2024-06-02T15:54:15.280 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.280 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo ./.. 2024-06-02T15:54:15.280 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfattr -n ceph.dir.layout.pool ./.. --only-values 2024-06-02T15:54:15.282 INFO:tasks.workunit.client.0.smithi090.stderr:./..: ceph.dir.layout.pool: No such attribute 2024-06-02T15:54:15.282 INFO:tasks.workunit.client.0.smithi090.stderr:+ datapool= 2024-06-02T15:54:15.282 INFO:tasks.workunit.client.0.smithi090.stderr:+ dir=./../.. 2024-06-02T15:54:15.282 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.283 INFO:tasks.workunit.client.0.smithi090.stdout:./../.. 2024-06-02T15:54:15.283 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo ./../.. 2024-06-02T15:54:15.283 INFO:tasks.workunit.client.0.smithi090.stderr:++ getfattr -n ceph.dir.layout.pool ./../.. --only-values 2024-06-02T15:54:15.286 INFO:tasks.workunit.client.0.smithi090.stderr:+ datapool=cephfs_data_ec 2024-06-02T15:54:15.286 INFO:tasks.workunit.client.0.smithi090.stderr:+ break 2024-06-02T15:54:15.286 INFO:tasks.workunit.client.0.smithi090.stderr:+ rm -f file file2 2024-06-02T15:54:15.288 INFO:tasks.workunit.client.0.smithi090.stderr:+ touch file file2 2024-06-02T15:54:15.293 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout file 2024-06-02T15:54:15.296 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.296 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-02T15:54:15.296 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout file 2024-06-02T15:54:15.297 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q object_size= 2024-06-02T15:54:15.300 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout file 2024-06-02T15:54:15.301 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q stripe_count= 2024-06-02T15:54:15.304 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout file 2024-06-02T15:54:15.304 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q stripe_unit= 2024-06-02T15:54:15.307 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout file 2024-06-02T15:54:15.307 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q pool= 2024-06-02T15:54:15.310 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool file 2024-06-02T15:54:15.312 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.313 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout.pool="cephfs_data_ec" 2024-06-02T15:54:15.313 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.313 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file 2024-06-02T15:54:15.316 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.316 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout.pool_namespace="" 2024-06-02T15:54:15.316 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.316 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit file 2024-06-02T15:54:15.318 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.318 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout.stripe_unit="4194304" 2024-06-02T15:54:15.318 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.319 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file 2024-06-02T15:54:15.321 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.321 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout.stripe_count="1" 2024-06-02T15:54:15.321 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.321 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size file 2024-06-02T15:54:15.324 INFO:tasks.workunit.client.0.smithi090.stdout:# file: file 2024-06-02T15:54:15.324 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.file.layout.object_size="4194304" 2024-06-02T15:54:15.324 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.325 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.bogus file 2024-06-02T15:54:15.326 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 'No such attribute' 2024-06-02T15:54:15.329 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout file 2024-06-02T15:54:15.329 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 'No such attribute' 2024-06-02T15:54:15.332 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2 2024-06-02T15:54:15.341 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout.stripe_count -v 8 file2 2024-06-02T15:54:15.347 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout.object_size -v 10485760 file2 2024-06-02T15:54:15.355 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout.pool -v cephfs_data_ec file2 2024-06-02T15:54:15.365 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-02T15:54:15.366 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.370 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout.pool_namespace -v foons file2 2024-06-02T15:54:15.377 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.377 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q foons 2024-06-02T15:54:15.380 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -x ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.387 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.387 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q -v foons 2024-06-02T15:54:15.391 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-02T15:54:15.392 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 1048576 2024-06-02T15:54:15.396 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.397 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 8 2024-06-02T15:54:15.401 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-02T15:54:15.401 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 10485760 2024-06-02T15:54:15.404 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=4194304 stripe_count=16 object_size=41943040 pool=cephfs_data_ec pool_namespace=foons' file2 2024-06-02T15:54:15.412 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-02T15:54:15.413 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 4194304 2024-06-02T15:54:15.417 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.418 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 16 2024-06-02T15:54:15.422 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-02T15:54:15.423 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 41943040 2024-06-02T15:54:15.427 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-02T15:54:15.428 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.432 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.432 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q foons 2024-06-02T15:54:15.435 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout -v stripe_unit=1048576 file2 2024-06-02T15:54:15.444 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-02T15:54:15.444 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 1048576 2024-06-02T15:54:15.448 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.448 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 16 2024-06-02T15:54:15.453 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-02T15:54:15.453 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 41943040 2024-06-02T15:54:15.455 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-02T15:54:15.456 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.460 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.461 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q foons 2024-06-02T15:54:15.464 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=2097152 stripe_count=4 object_size=2097152 pool=cephfs_data_ec pool_namespace=barns' file2 2024-06-02T15:54:15.471 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-06-02T15:54:15.471 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 2097152 2024-06-02T15:54:15.475 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.475 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 4 2024-06-02T15:54:15.479 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-06-02T15:54:15.479 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 2097152 2024-06-02T15:54:15.483 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-06-02T15:54:15.483 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.488 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-06-02T15:54:15.488 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q barns 2024-06-02T15:54:15.491 INFO:tasks.workunit.client.0.smithi090.stderr:+ rm -f dir/file 2024-06-02T15:54:15.494 INFO:tasks.workunit.client.0.smithi090.stderr:+ rmdir dir 2024-06-02T15:54:15.497 INFO:tasks.workunit.client.0.smithi090.stderr:rmdir: failed to remove 'dir': No such file or directory 2024-06-02T15:54:15.498 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.498 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p dir 2024-06-02T15:54:15.504 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -d -m - dir 2024-06-02T15:54:15.504 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q ceph.dir.layout 2024-06-02T15:54:15.507 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.507 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -d -m - dir 2024-06-02T15:54:15.508 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q ceph.file.layout 2024-06-02T15:54:15.510 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.510 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.513 INFO:tasks.workunit.client.0.smithi090.stderr:dir: ceph.dir.layout: No such attribute 2024-06-02T15:54:15.513 INFO:tasks.workunit.client.0.smithi090.stderr:+ true 2024-06-02T15:54:15.513 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.dir.layout.stripe_unit -v 1048576 dir 2024-06-02T15:54:15.521 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.dir.layout.stripe_count -v 8 dir 2024-06-02T15:54:15.527 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.dir.layout.object_size -v 10485760 dir 2024-06-02T15:54:15.534 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.dir.layout.pool -v cephfs_data_ec dir 2024-06-02T15:54:15.539 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.dir.layout.pool_namespace -v dirns dir 2024-06-02T15:54:15.545 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.549 INFO:tasks.workunit.client.0.smithi090.stdout:# file: dir 2024-06-02T15:54:15.549 INFO:tasks.workunit.client.0.smithi090.stdout:ceph.dir.layout="stripe_unit=1048576 stripe_count=8 object_size=10485760 pool=cephfs_data_ec pool_namespace=dirns" 2024-06-02T15:54:15.549 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-06-02T15:54:15.550 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.550 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q object_size=10485760 2024-06-02T15:54:15.556 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.556 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q stripe_count=8 2024-06-02T15:54:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:15 smithi120 ceph-mon[36451]: pgmap v808: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-06-02T15:54:15.559 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.560 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q stripe_unit=1048576 2024-06-02T15:54:15.563 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.564 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q pool=cephfs_data_ec 2024-06-02T15:54:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:15 smithi135 ceph-mon[36839]: pgmap v808: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-06-02T15:54:15.568 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.569 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q pool_namespace=dirns 2024-06-02T15:54:15.573 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout.pool dir 2024-06-02T15:54:15.574 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.577 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout.stripe_unit dir 2024-06-02T15:54:15.577 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 1048576 2024-06-02T15:54:15.580 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout.stripe_count dir 2024-06-02T15:54:15.581 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 8 2024-06-02T15:54:15.584 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout.object_size dir 2024-06-02T15:54:15.584 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 10485760 2024-06-02T15:54:15.587 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout.pool_namespace dir 2024-06-02T15:54:15.587 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q dirns 2024-06-02T15:54:15.590 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout -v stripe_count=16 file2 2024-06-02T15:54:15.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.595 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 16 2024-06-02T15:54:15.601 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -n ceph.file.layout -v 'object_size=10485760 stripe_count=8 stripe_unit=1048576 pool=cephfs_data_ec pool_namespace=dirns' file2 2024-06-02T15:54:15.609 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-06-02T15:54:15.609 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 8 2024-06-02T15:54:15.612 INFO:tasks.workunit.client.0.smithi090.stderr:+ touch dir/file 2024-06-02T15:54:15.616 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool dir/file 2024-06-02T15:54:15.616 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q cephfs_data_ec 2024-06-02T15:54:15.619 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_unit dir/file 2024-06-02T15:54:15.620 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 1048576 2024-06-02T15:54:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:15 smithi090 ceph-mon[31595]: pgmap v808: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-06-02T15:54:15.623 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.stripe_count dir/file 2024-06-02T15:54:15.624 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 8 2024-06-02T15:54:15.627 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.object_size dir/file 2024-06-02T15:54:15.627 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 10485760 2024-06-02T15:54:15.631 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.file.layout.pool_namespace dir/file 2024-06-02T15:54:15.631 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q dirns 2024-06-02T15:54:15.633 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -x ceph.dir.layout.pool_namespace dir 2024-06-02T15:54:15.642 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.642 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q -v pool_namespace=dirns 2024-06-02T15:54:15.645 INFO:tasks.workunit.client.0.smithi090.stderr:+ setfattr -x ceph.dir.layout dir 2024-06-02T15:54:15.650 INFO:tasks.workunit.client.0.smithi090.stderr:+ getfattr -n ceph.dir.layout dir 2024-06-02T15:54:15.651 INFO:tasks.workunit.client.0.smithi090.stderr:+ grep -q 'No such attribute' 2024-06-02T15:54:15.653 INFO:tasks.workunit.client.0.smithi090.stdout:OK 2024-06-02T15:54:15.654 INFO:tasks.workunit.client.0.smithi090.stderr:+ echo OK 2024-06-02T15:54:15.654 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:54:15.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:54:15.681 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:15.771 INFO:tasks.workunit:Running workunit fs/misc/mkpool_layout_vxattrs.sh... 2024-06-02T15:54:15.772 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/mkpool_layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/mkpool_layout_vxattrs.sh 2024-06-02T15:54:15.986 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 -- 172.21.15.90:0/2234306385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 msgr2=0x7fa1ec111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:15.986 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/2234306385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec111520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009ca0 tx=0x7fa1e402a8c0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.986 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 -- 172.21.15.90:0/2234306385 shutdown_connections 2024-06-02T15:54:15.986 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/2234306385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec111520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.986 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/2234306385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.987 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/2234306385 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec1090e0 0x7fa1ec109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.987 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 -- 172.21.15.90:0/2234306385 >> 172.21.15.90:0/2234306385 conn(0x7fa1ec0780a0 msgr2=0x7fa1ec07a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:15.987 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 -- 172.21.15.90:0/2234306385 shutdown_connections 2024-06-02T15:54:15.987 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.989+0000 7fa1f1e00640 1 -- 172.21.15.90:0/2234306385 wait complete. 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 Processor -- start 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 -- start start 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec1090e0 0x7fa1ec19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec1a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec114c20 con 0x7fa1ec10ac90 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec114aa0 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.990+0000 7fa1f1e00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec114da0 con 0x7fa1ec1090e0 2024-06-02T15:54:15.989 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec1a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec1a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55176/0 (socket says 172.21.15.90:55176) 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1eb400640 1 -- 172.21.15.90:0/3406478411 learned_addr learned my addr 172.21.15.90:0/3406478411 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1eaa00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec1090e0 0x7fa1ec19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 -- 172.21.15.90:0/3406478411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec1090e0 msgr2=0x7fa1ec19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec1090e0 0x7fa1ec19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 -- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 msgr2=0x7fa1ec1a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec1a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.991+0000 7fa1e2000640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec1a6ea0 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1e2000640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec19f7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9f0 tx=0x7fa1d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400ccc0 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1a7190 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d40108e0 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.990 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4018880 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.991 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.992+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a7730 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.992 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.996+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa1d4010430 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.994 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.997+0000 7fa1f0a00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 0x7fa1d00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:15.994 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.997+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9179+0+0 (secure 0 0 0) 0x7fa1d40e28e0 con 0x7fa1ec10a2d0 2024-06-02T15:54:15.994 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.997+0000 7fa1eaa00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 0x7fa1d00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:16.013 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:15.993+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec110480 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.014 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.018+0000 7fa1eaa00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 0x7fa1d00b4d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec10a130 tx=0x7fa1dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:16.019 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.022+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1d40a9ba0 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.236 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.239+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8} v 0) -- 0x7fa1ec02dba0 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.292 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.295+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]=0 pool 'foo.203833' created v91) ==== 123+0+0 (secure 0 0 0) 0x7fa1d40adea0 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:16 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:16 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.427+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8} v 0) -- 0x7fa1ec02def0 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.425 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.428+0000 7fa1f0a00640 1 -- 172.21.15.90:0/3406478411 <== mon.2 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]=0 pool 'foo.203833' already exists v91) ==== 130+0+0 (secure 0 0 0) 0x7fa1d40b2950 con 0x7fa1ec10a2d0 2024-06-02T15:54:16.425 INFO:tasks.workunit.client.0.smithi090.stderr:pool 'foo.203833' already exists 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 msgr2=0x7fa1d00b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 0x7fa1d00b4d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec10a130 tx=0x7fa1dc005f90 comp rx=0 tx=0).stop 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 msgr2=0x7fa1ec19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec19f7c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400e9f0 tx=0x7fa1d400eeb0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 shutdown_connections 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa1d00b2870 0x7fa1d00b4d30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.428 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1ec10ac90 0x7fa1ec1a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.429 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1ec10a2d0 0x7fa1ec19f7c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.429 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 --2- 172.21.15.90:0/3406478411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1ec1090e0 0x7fa1ec19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.429 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 >> 172.21.15.90:0/3406478411 conn(0x7fa1ec0780a0 msgr2=0x7fa1ec10d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:16.429 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 shutdown_connections 2024-06-02T15:54:16.429 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.431+0000 7fa1f1e00640 1 -- 172.21.15.90:0/3406478411 wait complete. 2024-06-02T15:54:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:16 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:16 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 -- 172.21.15.90:0/3205610719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10a180 msgr2=0x7f31ec10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 --2- 172.21.15.90:0/3205610719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10a180 0x7f31ec10a600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009900 tx=0x7f31d802a410 comp rx=0 tx=0).stop 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 -- 172.21.15.90:0/3205610719 shutdown_connections 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 --2- 172.21.15.90:0/3205610719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec10ab40 0x7f31ec1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 --2- 172.21.15.90:0/3205610719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10a180 0x7f31ec10a600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 --2- 172.21.15.90:0/3205610719 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec108f90 0x7f31ec1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 -- 172.21.15.90:0/3205610719 >> 172.21.15.90:0/3205610719 conn(0x7f31ec077f40 msgr2=0x7f31ec07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 -- 172.21.15.90:0/3205610719 shutdown_connections 2024-06-02T15:54:16.564 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.567+0000 7f31ebe00640 1 -- 172.21.15.90:0/3205610719 wait complete. 2024-06-02T15:54:16.566 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 Processor -- start 2024-06-02T15:54:16.566 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 -- start start 2024-06-02T15:54:16.566 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec108f90 0x7f31ec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 0x7f31ec19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 0x7f31ec1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec114a60 con 0x7f31ec10ab40 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec1148e0 con 0x7f31ec10a180 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.568+0000 7f31ebe00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31ec114be0 con 0x7f31ec108f90 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 0x7f31ec1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 0x7f31ec19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 0x7f31ec1a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55192/0 (socket says 172.21.15.90:55192) 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31eb400640 1 -- 172.21.15.90:0/2686709958 learned_addr learned my addr 172.21.15.90:0/2686709958 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31eaa00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec108f90 0x7f31ec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 -- 172.21.15.90:0/2686709958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec108f90 msgr2=0x7f31ec19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec108f90 0x7f31ec19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.567 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 -- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 msgr2=0x7f31ec1a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 0x7f31ec1a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ec1a6cb0 con 0x7f31ec10a180 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.569+0000 7f31ea000640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 0x7f31ec19f5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006010 tx=0x7f31d80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.570+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d803f030 con 0x7f31ec10a180 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.570+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31d8031910 con 0x7f31ec10a180 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.570+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ec1a6f40 con 0x7f31ec10a180 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.570+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8035070 con 0x7f31ec10a180 2024-06-02T15:54:16.568 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.570+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ec1a73b0 con 0x7f31ec10a180 2024-06-02T15:54:16.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:16 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.569 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:16 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:16.569 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.571+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ec1102b0 con 0x7f31ec10a180 2024-06-02T15:54:16.570 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.573+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f31d8042030 con 0x7f31ec10a180 2024-06-02T15:54:16.571 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.575+0000 7f31e3400640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 0x7f31d00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:16.572 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.575+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9528+0+0 (secure 0 0 0) 0x7f31d8102e20 con 0x7f31ec10a180 2024-06-02T15:54:16.572 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.575+0000 7f31eaa00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 0x7f31d00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:16.572 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.576+0000 7f31eaa00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 0x7f31d00b4de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31ec109fe0 tx=0x7f31dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:16.573 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.577+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31d80c9f80 con 0x7f31ec10a180 2024-06-02T15:54:16.860 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:16.864+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"} v 0) -- 0x7f31ec02dba0 con 0x7f31ec10a180 2024-06-02T15:54:17.301 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.304+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]=0 added data pool 5 to fsmap v38) ==== 133+0+0 (secure 0 0 0) 0x7f31d80ce280 con 0x7f31ec10a180 2024-06-02T15:54:17.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.479+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"} v 0) -- 0x7f31ec0636f0 con 0x7f31ec10a180 2024-06-02T15:54:17.478 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.481+0000 7f31e3400640 1 -- 172.21.15.90:0/2686709958 <== mon.2 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]=-22 RADOS pool 'foo.203833' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' v38) ==== 211+0+0 (secure 0 0 0) 0x7f31d80d2d30 con 0x7f31ec10a180 2024-06-02T15:54:17.478 INFO:tasks.workunit.client.0.smithi090.stderr:Second attempt of previously successful command failed with EINVAL: RADOS pool 'foo.203833' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-06-02T15:54:17.478 INFO:tasks.workunit.client.0.smithi090.stderr:RADOS pool 'foo.203833' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-06-02T15:54:17.480 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.483+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 msgr2=0x7f31d00b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:17.480 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.483+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 0x7f31d00b4de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f31ec109fe0 tx=0x7f31dc005f90 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.483+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 msgr2=0x7f31ec19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.483+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 0x7f31ec19f5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006010 tx=0x7f31d80042f0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 shutdown_connections 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f31d00b2920 0x7f31d00b4de0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31ec10ab40 0x7f31ec1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ec10a180 0x7f31ec19f5d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 --2- 172.21.15.90:0/2686709958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31ec108f90 0x7f31ec19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 >> 172.21.15.90:0/2686709958 conn(0x7f31ec077f40 msgr2=0x7f31ec10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 shutdown_connections 2024-06-02T15:54:17.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.484+0000 7f31ebe00640 1 -- 172.21.15.90:0/2686709958 wait complete. 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: pgmap v809: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]': finished 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:17 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: pgmap v809: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]': finished 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:17 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: pgmap v809: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 19 op/s 2024-06-02T15:54:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]': finished 2024-06-02T15:54:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T15:54:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3406478411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.203833", "pg_num": 8}]: dispatch 2024-06-02T15:54:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:17 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 -- 172.21.15.90:0/1167473762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 msgr2=0x7faca410d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 --2- 172.21.15.90:0/1167473762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca410d110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009be0 tx=0x7fac9c02a6d0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 -- 172.21.15.90:0/1167473762 shutdown_connections 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 --2- 172.21.15.90:0/1167473762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca410d110 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 --2- 172.21.15.90:0/1167473762 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4105ec0 0x7faca4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 --2- 172.21.15.90:0/1167473762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4104cd0 0x7faca41050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 -- 172.21.15.90:0/1167473762 >> 172.21.15.90:0/1167473762 conn(0x7faca4100240 msgr2=0x7faca4102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:17.627 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 -- 172.21.15.90:0/1167473762 shutdown_connections 2024-06-02T15:54:17.628 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.630+0000 7faca9000640 1 -- 172.21.15.90:0/1167473762 wait complete. 2024-06-02T15:54:17.628 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 Processor -- start 2024-06-02T15:54:17.628 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 -- start start 2024-06-02T15:54:17.629 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:17.629 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 0x7faca419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca41a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faca41107e0 con 0x7faca4105ec0 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faca4110660 con 0x7faca4104cd0 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca9000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faca4110960 con 0x7faca4106880 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.632+0000 7faca1e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 0x7faca419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35558/0 (socket says 172.21.15.90:35558) 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 -- 172.21.15.90:0/1297638407 learned_addr learned my addr 172.21.15.90:0/1297638407 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca2800640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca41a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 -- 172.21.15.90:0/1297638407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 msgr2=0x7faca41a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:17.630 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca41a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.631 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 -- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 msgr2=0x7faca419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:17.631 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 0x7faca419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:17.631 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41a2ad0 con 0x7faca4104cd0 2024-06-02T15:54:17.631 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1400640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 0x7faca419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7faca1e00640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e9a0 tx=0x7fac9400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.633+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9400cd80 con 0x7faca4104cd0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.634+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac940108e0 con 0x7faca4104cd0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.634+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94018960 con 0x7faca4104cd0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.634+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca41a2dc0 con 0x7faca4104cd0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.634+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41a3230 con 0x7faca4104cd0 2024-06-02T15:54:17.632 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.635+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca410c090 con 0x7faca4104cd0 2024-06-02T15:54:17.634 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.637+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac94010a90 con 0x7faca4104cd0 2024-06-02T15:54:17.635 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.639+0000 7fac9aa00640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 0x7fac880b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:17.636 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.639+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9560+0+0 (secure 0 0 0) 0x7fac940e3120 con 0x7faca4104cd0 2024-06-02T15:54:17.636 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.639+0000 7faca1400640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 0x7fac880b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:17.636 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.640+0000 7faca1400640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 0x7fac880b4d30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faca419c3c0 tx=0x7fac90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:17.637 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.641+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac94026070 con 0x7faca4104cd0 2024-06-02T15:54:17.894 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:17.898+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"} v 0) -- 0x7faca402de50 con 0x7faca4104cd0 2024-06-02T15:54:18.302 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:18.306+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]=0 removed data pool 5 from fsmap v39) ==== 136+0+0 (secure 0 0 0) 0x7fac940aa8b0 con 0x7faca4104cd0 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:18 smithi090 ceph-mon[31595]: pgmap v812: 137 pgs: 1 creating+activating, 6 unknown, 130 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 -- 172.21.15.90:0/3910660378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8075770 msgr2=0x7f0ae8075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3910660378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8075770 0x7f0ae8075b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009990 tx=0x7f0adc009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 -- 172.21.15.90:0/3910660378 shutdown_connections 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3910660378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 0x7f0ae8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3910660378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8075770 0x7f0ae8075b50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3910660378 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8076090 0x7f0ae810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 -- 172.21.15.90:0/3910660378 >> 172.21.15.90:0/3910660378 conn(0x7f0ae807a6a0 msgr2=0x7f0ae807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 -- 172.21.15.90:0/3910660378 shutdown_connections 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.430+0000 7f0aefa00700 1 -- 172.21.15.90:0/3910660378 wait complete. 2024-06-02T15:54:18.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.431+0000 7f0aefa00700 1 Processor -- start 2024-06-02T15:54:18.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.431+0000 7f0aefa00700 1 -- start start 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.431+0000 7f0aefa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 0x7f0ae819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aefa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aefa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 0x7f0ae81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aefa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8115940 con 0x7f0ae8076090 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aefa00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81157c0 con 0x7f0ae8075770 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aefa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae8115ac0 con 0x7f0ae810ff40 2024-06-02T15:54:18.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55228/0 (socket says 172.21.15.90:55228) 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0ae7e00700 1 -- 172.21.15.90:0/3917773130 learned_addr learned my addr 172.21.15.90:0/3917773130 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 0x7f0ae819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aeda00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 0x7f0ae81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 -- 172.21.15.90:0/3917773130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 msgr2=0x7f0ae81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 0x7f0ae81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 -- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 msgr2=0x7f0ae819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.432+0000 7f0aed000700 1 -- 172.21.15.90:0/3917773130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae81a0f00 con 0x7f0ae8075770 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0aed000700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 0x7f0ae819bef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0adc000c00 tx=0x7f0adc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0ae5600700 1 -- 172.21.15.90:0/3917773130 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc026070 con 0x7f0ae8075770 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae81a1190 con 0x7f0ae8075770 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0ae5600700 1 -- 172.21.15.90:0/3917773130 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc021410 con 0x7f0ae8075770 2024-06-02T15:54:18.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0ae5600700 1 -- 172.21.15.90:0/3917773130 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc017650 con 0x7f0ae8075770 2024-06-02T15:54:18.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.433+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae81a8a70 con 0x7f0ae8075770 2024-06-02T15:54:18.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.437+0000 7f0ae5600700 1 -- 172.21.15.90:0/3917773130 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0adc029070 con 0x7f0ae8075770 2024-06-02T15:54:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.438+0000 7f0ae5600700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 0x7f0ad00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.439+0000 7f0ae5600700 1 -- 172.21.15.90:0/3917773130 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9560+0+0 (secure 0 0 0) 0x7f0adc0e3530 con 0x7f0ae8075770 2024-06-02T15:54:18.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.439+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 0x7f0ad00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.440+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 0x7f0ad00b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8111c40 tx=0x7f0ad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.440+0000 7f0afa450b80 1 Processor -- start 2024-06-02T15:54:18.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.440+0000 7f0afa450b80 1 -- start start 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d3cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d3d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582d5c9ae40 con 0x5582d5d42610 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5582d5c8a9e0 con 0x5582d5d3d140 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0afa450b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5582d5c8ea20 con 0x5582d5d42f30 2024-06-02T15:54:18.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0aed000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0aed000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55236/0 (socket says 172.21.15.90:55236) 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0aed000700 1 -- 172.21.15.90:0/1726835137 learned_addr learned my addr 172.21.15.90:0/1726835137 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d3cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.441+0000 7f0aeda00700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d3d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 -- 172.21.15.90:0/1726835137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 msgr2=0x5582d5d3cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d3cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 -- 172.21.15.90:0/1726835137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 msgr2=0x5582d5d3d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d3d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 -- 172.21.15.90:0/1726835137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5c53420 con 0x5582d5d42610 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aeda00700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d3d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d3cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0aed000700 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa5e0 tx=0x7f0adc0aa550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e79f73518ef8a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.442+0000 7f0ad6000700 1 -- 172.21.15.90:0/1726835137 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc0aea00 con 0x5582d5d42610 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.443+0000 7f0ad6000700 1 -- 172.21.15.90:0/1726835137 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc0b3420 con 0x5582d5d42610 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.443+0000 7f0ad6000700 1 -- 172.21.15.90:0/1726835137 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc0e3120 con 0x5582d5d42610 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.443+0000 7f0afa450b80 1 -- 172.21.15.90:0/1726835137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 msgr2=0x5582d5d429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.443+0000 7f0afa450b80 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa5e0 tx=0x7f0adc0aa550 comp rx=0 tx=0).stop 2024-06-02T15:54:18.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.443+0000 7f0afa450b80 1 -- 172.21.15.90:0/1726835137 shutdown_connections 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d3d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d429f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 --2- 172.21.15.90:0/1726835137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d3cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 -- 172.21.15.90:0/1726835137 >> 172.21.15.90:0/1726835137 conn(0x5582d5c6abe0 msgr2=0x5582d5d37690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 -- 172.21.15.90:0/1726835137 shutdown_connections 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 -- 172.21.15.90:0/1726835137 wait complete. 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 -- start start 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.444+0000 7f0afa450b80 1 Processor -- start 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d48cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582d5b90fa0 con 0x5582d5d42610 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5582d5c2b760 con 0x5582d5d3d140 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5582d5d497a0 con 0x5582d5d42f30 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0afa450b80 10 client.0 _ll_register_callbacks cb 0x5582d5c34c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0aed000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d48cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55238/0 (socket says 172.21.15.90:55238) 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.445+0000 7f0ae7e00700 1 -- 172.21.15.90:0/2596632734 learned_addr learned my addr 172.21.15.90:0/2596632734 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0aeda00700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 -- 172.21.15.90:0/2596632734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 msgr2=0x5582d5d465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 -- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 msgr2=0x5582d5d48cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d48cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 -- 172.21.15.90:0/2596632734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d5bda8d0 con 0x5582d5d42610 2024-06-02T15:54:18.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0aed000700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d48cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800ed70 tx=0x7f0ad80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac35bceac737591e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.446+0000 7f0acaa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad801db00 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0adc0b3420 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9560+0+0 (secure 0 0 0) 0x7f0ad801dca0 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ad8031050 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0afa450b80 10 client.16218 fetch_fsmap learned FSMap version 39 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.447+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5582d5c5b060 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc0b3420 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0ad80054f0 con 0x5582d5d42610 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 10 client.16218 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 20 client.16218 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5d562c0 0x5582d5d58730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 4 client.16218 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582d5c9cd90 con 0x5582d5d562c0 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.448+0000 7f0aed000700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5d562c0 0x5582d5d58730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.449+0000 7f0aed000700 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5d562c0 0x5582d5d58730 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4929b34c7ccf4b42 server_cookie=b9fa27c144aa4e64 in_seq=0 out_seq=0 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.449+0000 7f0acaa00700 10 client.16218 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0acaa00700 1 -- 172.21.15.90:0/2596632734 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5582d5c9cd90 con 0x5582d5d562c0 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0acaa00700 10 client.16218 handle_command_reply: tid=0 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 client.16218 shutdown 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 msgr2=0x5582d5d460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800ed70 tx=0x7f0ad80047c0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 shutdown_connections 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5d562c0 0x5582d5d58730 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42f30 0x5582d5d465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d48cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 --2- 172.21.15.90:0/2596632734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42610 0x5582d5d460b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 >> 172.21.15.90:0/2596632734 conn(0x5582d5c6abe0 msgr2=0x5582d5d36d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0acaa00700 0 client.16218 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:18.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 shutdown_connections 2024-06-02T15:54:18.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.450+0000 7f0afa450b80 1 -- 172.21.15.90:0/2596632734 wait complete. 2024-06-02T15:54:18.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.451+0000 7f0afa450b80 20 client.16218 trim_cache size 0 max 16384 2024-06-02T15:54:18.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.467+0000 7f0afa450b80 1 Processor -- start 2024-06-02T15:54:18.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.467+0000 7f0afa450b80 1 -- start start 2024-06-02T15:54:18.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 0x5582d5d466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 0x5582d5d3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81b45b0 con 0x5582d5d42f30 2024-06-02T15:54:18.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81b1d80 con 0x5582d5d42610 2024-06-02T15:54:18.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0afa450b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81afbd0 con 0x5582d5d3d140 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0aeda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0aeda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55240/0 (socket says 172.21.15.90:55240) 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0aed000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 0x5582d5d466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0aeda00700 1 -- 172.21.15.90:0/611134053 learned_addr learned my addr 172.21.15.90:0/611134053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.468+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 0x5582d5d3d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 -- 172.21.15.90:0/611134053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 msgr2=0x5582d5d3d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 0x5582d5d3d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 -- 172.21.15.90:0/611134053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 msgr2=0x5582d5d43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 -- 172.21.15.90:0/611134053 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d5d58db0 con 0x5582d5d3d140 2024-06-02T15:54:18.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 0x5582d5d3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aeda00700 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0aed000700 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 0x5582d5d466e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa5e0 tx=0x7f0adc00b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=520f88edab90b9a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0ad5600700 1 -- 172.21.15.90:0/611134053 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc092110 con 0x5582d5d3d140 2024-06-02T15:54:18.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0ad5600700 1 -- 172.21.15.90:0/611134053 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc0218c0 con 0x5582d5d3d140 2024-06-02T15:54:18.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.469+0000 7f0ad5600700 1 -- 172.21.15.90:0/611134053 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc0aa8b0 con 0x5582d5d3d140 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 -- 172.21.15.90:0/611134053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 msgr2=0x5582d5d466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 0x5582d5d466e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0adc0aa5e0 tx=0x7f0adc00b4b0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 -- 172.21.15.90:0/611134053 shutdown_connections 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d3d140 0x5582d5d466e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 --2- 172.21.15.90:0/611134053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d42610 0x5582d5d3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 -- 172.21.15.90:0/611134053 >> 172.21.15.90:0/611134053 conn(0x5582d5c6abe0 msgr2=0x5582d5d449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 -- 172.21.15.90:0/611134053 shutdown_connections 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.470+0000 7f0afa450b80 1 -- 172.21.15.90:0/611134053 wait complete. 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.471+0000 7f0afa450b80 1 -- start start 2024-06-02T15:54:18.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.471+0000 7f0afa450b80 1 Processor -- start 2024-06-02T15:54:18.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d5d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81b0240 con 0x5582d5d42f30 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81b16b0 con 0x5582d5d3d140 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81b0910 con 0x5582d5d42610 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0afa450b80 10 client.0 _ll_register_callbacks cb 0x5582d5c34c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0aed000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:49726/0 (socket says 172.21.15.90:49726) 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 -- 172.21.15.90:0/474537222 learned_addr learned my addr 172.21.15.90:0/474537222 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0aeda00700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d5d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 -- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 msgr2=0x5582d5d4e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d4e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 -- 172.21.15.90:0/474537222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 msgr2=0x5582d5d5d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d5d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0ae7e00700 1 -- 172.21.15.90:0/474537222 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d5d5d610 con 0x5582d5d42610 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0aed000700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d4e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.472+0000 7f0aeda00700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d5d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0ae7e00700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8026490 tx=0x7f0ad8026930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1fc2720bb1edd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:18.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0acaa00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8026d20 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0adc0218c0 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9560+0+0 (secure 0 0 0) 0x7f0ad801d9d0 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0ad8045050 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0afa450b80 10 client.26458 fetch_fsmap learned FSMap version 39 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.473+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5582d5dc0770 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc0218c0 con 0x5582d5d42610 2024-06-02T15:54:18.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f0ad8037b30 con 0x5582d5d42610 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 10 client.26458 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 20 client.26458 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5dc0e20 0x5582d5dc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 4 client.26458 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.474+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582d5c96570 con 0x5582d5dc0e20 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.475+0000 7f0aed000700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5dc0e20 0x5582d5dc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.475+0000 7f0acaa00700 10 client.26458 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.475+0000 7f0aed000700 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5dc0e20 0x5582d5dc3240 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d9f942319082c5 server_cookie=c2796500d5c9cb9d in_seq=0 out_seq=0 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.475+0000 7f0acaa00700 1 -- 172.21.15.90:0/474537222 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582d5c96570 con 0x5582d5dc0e20 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0acaa00700 10 client.26458 handle_command_reply: tid=0 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 client.26458 shutdown 2024-06-02T15:54:18.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 msgr2=0x5582d5d4ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8026490 tx=0x7f0ad8026930 comp rx=0 tx=0).stop 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 shutdown_connections 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5582d5dc0e20 0x5582d5dc3240 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582d5d42f30 0x5582d5d5d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5582d5d3d140 0x5582d5d4e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 --2- 172.21.15.90:0/474537222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582d5d42610 0x5582d5d4ef30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0acaa00700 0 client.26458 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 >> 172.21.15.90:0/474537222 conn(0x5582d5c6abe0 msgr2=0x5582d5d45230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 shutdown_connections 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 1 -- 172.21.15.90:0/474537222 wait complete. 2024-06-02T15:54:18.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.476+0000 7f0afa450b80 20 client.26458 trim_cache size 0 max 16384 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 msgr2=0x7f0ad00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 0x7f0ad00b4d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8111c40 tx=0x7f0ad8009260 comp rx=0 tx=0).stop 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 msgr2=0x7f0ae819bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 0x7f0ae819bef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0adc000c00 tx=0x7f0adc0047b0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 shutdown_connections 2024-06-02T15:54:18.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ad00b2880 0x7f0ad00b4d40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae810ff40 0x7f0ae81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ae8075770 0x7f0ae819bef0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 --2- 172.21.15.90:0/3917773130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ae8076090 0x7f0ae819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.477+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 >> 172.21.15.90:0/3917773130 conn(0x7f0ae807a6a0 msgr2=0x7f0ae8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.478+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 shutdown_connections 2024-06-02T15:54:18.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:18.478+0000 7f0aefa00700 1 -- 172.21.15.90:0/3917773130 wait complete. 2024-06-02T15:54:18.513 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:18.515+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"} v 0) -- 0x7faca4063650 con 0x7faca4104cd0 2024-06-02T15:54:18.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.558 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.559 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:18 smithi120 ceph-mon[36451]: pgmap v812: 137 pgs: 1 creating+activating, 6 unknown, 130 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-02T15:54:18.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T15:54:18.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:18.567 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:18.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2686709958' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.568 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:18.571 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:18 smithi135 ceph-mon[36839]: pgmap v812: 137 pgs: 1 creating+activating, 6 unknown, 130 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-02T15:54:19.317 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.320+0000 7fac9aa00640 1 -- 172.21.15.90:0/1297638407 <== mon.2 v2:172.21.15.120:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]=0 v40) ==== 106+0+0 (secure 0 0 0) 0x7fac940aa1b0 con 0x7faca4104cd0 2024-06-02T15:54:19.321 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.324+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 msgr2=0x7fac880b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:19.321 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.324+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 0x7fac880b4d30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faca419c3c0 tx=0x7fac90005f90 comp rx=0 tx=0).stop 2024-06-02T15:54:19.321 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.324+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 msgr2=0x7faca419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.324+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e9a0 tx=0x7fac9400ee60 comp rx=0 tx=0).stop 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 shutdown_connections 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fac880b2870 0x7fac880b4d30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faca4106880 0x7faca41a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faca4105ec0 0x7faca419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 --2- 172.21.15.90:0/1297638407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faca4104cd0 0x7faca419aeb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 >> 172.21.15.90:0/1297638407 conn(0x7faca4100240 msgr2=0x7faca4109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 shutdown_connections 2024-06-02T15:54:19.322 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.325+0000 7faca9000640 1 -- 172.21.15.90:0/1297638407 wait complete. 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 -- 172.21.15.90:0/1551988748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c105ed0 msgr2=0x7f019c106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 --2- 172.21.15.90:0/1551988748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c105ed0 0x7f019c106350 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0184009900 tx=0x7f018402a400 comp rx=0 tx=0).stop 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 -- 172.21.15.90:0/1551988748 shutdown_connections 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 --2- 172.21.15.90:0/1551988748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c106890 0x7f019c10d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 --2- 172.21.15.90:0/1551988748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c105ed0 0x7f019c106350 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 --2- 172.21.15.90:0/1551988748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c104ce0 0x7f019c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 -- 172.21.15.90:0/1551988748 >> 172.21.15.90:0/1551988748 conn(0x7f019c100270 msgr2=0x7f019c102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 -- 172.21.15.90:0/1551988748 shutdown_connections 2024-06-02T15:54:19.472 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.475+0000 7f01a1400640 1 -- 172.21.15.90:0/1551988748 wait complete. 2024-06-02T15:54:19.473 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.476+0000 7f01a1400640 1 Processor -- start 2024-06-02T15:54:19.473 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.476+0000 7f01a1400640 1 -- start start 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c106890 0x7f019c1a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f019c110860 con 0x7f019c104ce0 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f019c1106e0 con 0x7f019c106890 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f01a1400640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f019c1109e0 con 0x7f019c105ed0 2024-06-02T15:54:19.475 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f0193e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f019a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f019a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55258/0 (socket says 172.21.15.90:55258) 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f019a800640 1 -- 172.21.15.90:0/3048466873 learned_addr learned my addr 172.21.15.90:0/3048466873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f0193e00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:49748/0 (socket says 172.21.15.90:49748) 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.477+0000 7f019b200640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c106890 0x7f019c1a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 -- 172.21.15.90:0/3048466873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 msgr2=0x7f019c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 -- 172.21.15.90:0/3048466873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c106890 msgr2=0x7f019c1a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c106890 0x7f019c1a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c1a2b50 con 0x7f019c104ce0 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f0193e00640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f019a800640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f018c00e9c0 tx=0x7f018c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:19.476 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c00cd80 con 0x7f019c104ce0 2024-06-02T15:54:19.477 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018c0035f0 con 0x7f019c104ce0 2024-06-02T15:54:19.477 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c1a2e40 con 0x7f019c104ce0 2024-06-02T15:54:19.477 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c010690 con 0x7f019c104ce0 2024-06-02T15:54:19.477 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.478+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c1a32a0 con 0x7f019c104ce0 2024-06-02T15:54:19.477 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.479+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c10c0a0 con 0x7f019c104ce0 2024-06-02T15:54:19.478 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.482+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f018c0037a0 con 0x7f019c104ce0 2024-06-02T15:54:19.480 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.483+0000 7f0193400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 0x7f017c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:19.480 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.483+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9560+0+0 (secure 0 0 0) 0x7f018c0e2c20 con 0x7f019c104ce0 2024-06-02T15:54:19.480 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.484+0000 7f0193e00640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 0x7f017c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:19.481 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.484+0000 7f0193e00640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 0x7f017c0b4cb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0184031500 tx=0x7f0184005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:19.483 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.486+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f018c0aa3b0 con 0x7f019c104ce0 2024-06-02T15:54:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:19 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:19 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:19 smithi120 ceph-mon[36451]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T15:54:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:19 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:19 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:19 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:19 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:19 smithi135 ceph-mon[36839]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T15:54:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:19 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:19 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:19 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:19.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:19 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:19.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:19 smithi090 ceph-mon[31595]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T15:54:19.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:19 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1297638407' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:19 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]: dispatch 2024-06-02T15:54:19.696 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:19.699+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true} v 0) -- 0x7f019c02dba0 con 0x7f019c104ce0 2024-06-02T15:54:20.326 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.329+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]=0 pool 'foo.203833' removed v94) ==== 166+0+0 (secure 0 0 0) 0x7f018c0a9cb0 con 0x7f019c104ce0 2024-06-02T15:54:20.454 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.457+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true} v 0) -- 0x7f019c02de70 con 0x7f019c104ce0 2024-06-02T15:54:20.455 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.458+0000 7f0193400640 1 -- 172.21.15.90:0/3048466873 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]=0 pool 'foo.203833' does not exist v94) ==== 173+0+0 (secure 0 0 0) 0x7f018c0ae060 con 0x7f019c104ce0 2024-06-02T15:54:20.455 INFO:tasks.workunit.client.0.smithi090.stderr:pool 'foo.203833' does not exist 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 msgr2=0x7f017c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 0x7f017c0b4cb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0184031500 tx=0x7f0184005a20 comp rx=0 tx=0).stop 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 msgr2=0x7f019c19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f018c00e9c0 tx=0x7f018c00ee80 comp rx=0 tx=0).stop 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 shutdown_connections 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f017c0b27f0 0x7f017c0b4cb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f019c106890 0x7f019c1a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:20.458 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f019c105ed0 0x7f019c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:20.459 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 --2- 172.21.15.90:0/3048466873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f019c104ce0 0x7f019c19af30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:20.459 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 >> 172.21.15.90:0/3048466873 conn(0x7f019c100270 msgr2=0x7f019c109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:20.459 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 shutdown_connections 2024-06-02T15:54:20.459 INFO:tasks.workunit.client.0.smithi090.stderr:2024-06-02T15:54:20.461+0000 7f01a1400640 1 -- 172.21.15.90:0/3048466873 wait complete. 2024-06-02T15:54:20.472 INFO:tasks.workunit.client.0.smithi090.stdout:OK 2024-06-02T15:54:20.473 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T15:54:20.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T15:54:20.576 INFO:tasks.workunit:Running workunit fs/misc/multiple_rsync.sh... 2024-06-02T15:54:20.576 DEBUG:teuthology.orchestra.run.smithi090:workunit test fs/misc/multiple_rsync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh 2024-06-02T15:54:20.589 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:54:20.589 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:54:20.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:20 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:20 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:20 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:20 smithi090 ceph-mon[31595]: pgmap v814: 137 pgs: 1 creating+activating, 5 unknown, 131 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-06-02T15:54:20.654 INFO:tasks.workunit.client.0.smithi090.stderr:+ export PAYLOAD=/tmp/multiple_rsync_payload.204082 2024-06-02T15:54:20.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ PAYLOAD=/tmp/multiple_rsync_payload.204082 2024-06-02T15:54:20.655 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo cp -r /usr/lib/ /tmp/multiple_rsync_payload.204082 2024-06-02T15:54:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:20 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:20 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:20 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:20 smithi120 ceph-mon[36451]: pgmap v814: 137 pgs: 1 creating+activating, 5 unknown, 131 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-06-02T15:54:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:20 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.203833"}]': finished 2024-06-02T15:54:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:20 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:20 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:20 smithi135 ceph-mon[36839]: pgmap v814: 137 pgs: 1 creating+activating, 5 unknown, 131 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-06-02T15:54:21.035 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:21.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:21 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:54:21.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:21 smithi090 ceph-mon[31595]: osdmap e94: 12 total, 12 up, 12 in 2024-06-02T15:54:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:21 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:21 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:54:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:21 smithi120 ceph-mon[36451]: osdmap e94: 12 total, 12 up, 12 in 2024-06-02T15:54:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:21 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:21 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:54:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:21 smithi135 ceph-mon[36839]: osdmap e94: 12 total, 12 up, 12 in 2024-06-02T15:54:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:21 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3048466873' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.203833", "pool2": "foo.203833", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:54:22.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:22 smithi090 ceph-mon[31595]: pgmap v816: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-06-02T15:54:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-mon[36451]: pgmap v816: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-06-02T15:54:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:22 smithi135 ceph-mon[36839]: pgmap v816: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-06-02T15:54:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 -- 172.21.15.90:0/1505047838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a50105d00 msgr2=0x7f9a50106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 --2- 172.21.15.90:0/1505047838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a50105d00 0x7f9a50106180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009900 tx=0x7f9a3c02a400 comp rx=0 tx=0).stop 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 -- 172.21.15.90:0/1505047838 shutdown_connections 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 --2- 172.21.15.90:0/1505047838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a501066c0 0x7f9a5010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 --2- 172.21.15.90:0/1505047838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a50105d00 0x7f9a50106180 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 --2- 172.21.15.90:0/1505047838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a50104b10 0x7f9a50104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 -- 172.21.15.90:0/1505047838 >> 172.21.15.90:0/1505047838 conn(0x7f9a50100080 msgr2=0x7f9a501024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 -- 172.21.15.90:0/1505047838 shutdown_connections 2024-06-02T15:54:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.834+0000 7f9a56c00640 1 -- 172.21.15.90:0/1505047838 wait complete. 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 Processor -- start 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 -- start start 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a50104b10 0x7f9a5019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a50105d00 0x7f9a5019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a501066c0 0x7f9a501a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50110620 con 0x7f9a50104b10 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501104a0 con 0x7f9a501066c0 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.835+0000 7f9a56c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501107a0 con 0x7f9a50105d00 2024-06-02T15:54:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a50104b10 0x7f9a5019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a50104b10 0x7f9a5019ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44538/0 (socket says 172.21.15.90:44538) 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 -- 172.21.15.90:0/1707942831 learned_addr learned my addr 172.21.15.90:0/1707942831 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a56200640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a501066c0 0x7f9a501a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a54e00640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a50105d00 0x7f9a5019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 -- 172.21.15.90:0/1707942831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a50105d00 msgr2=0x7f9a5019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a50105d00 0x7f9a5019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 -- 172.21.15.90:0/1707942831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a501066c0 msgr2=0x7f9a501a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a501066c0 0x7f9a501a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a55800640 1 -- 172.21.15.90:0/1707942831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501a2970 con 0x7f9a50104b10 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.836+0000 7f9a56200640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9a501066c0 0x7f9a501a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a55800640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a50104b10 0x7f9a5019ad50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400e9c0 tx=0x7f9a4400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a4e000640 1 -- 172.21.15.90:0/1707942831 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ccc0 con 0x7f9a50104b10 2024-06-02T15:54:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a4e000640 1 -- 172.21.15.90:0/1707942831 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a440035f0 con 0x7f9a50104b10 2024-06-02T15:54:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a4e000640 1 -- 172.21.15.90:0/1707942831 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ce60 con 0x7f9a50104b10 2024-06-02T15:54:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a56c00640 1 -- 172.21.15.90:0/1707942831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a501a2c60 con 0x7f9a50104b10 2024-06-02T15:54:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.837+0000 7f9a56c00640 1 -- 172.21.15.90:0/1707942831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a501a3180 con 0x7f9a50104b10 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 Processor -- start 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 -- start start 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd21110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556e1bc90b90 con 0x556e1bd26850 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e1bc80730 con 0x556e1bd20cf0 2024-06-02T15:54:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.841+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e1bc84770 con 0x556e1bd216e0 2024-06-02T15:54:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a54e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a54e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44554/0 (socket says 172.21.15.90:44554) 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a54e00640 1 -- 172.21.15.90:0/3352227618 learned_addr learned my addr 172.21.15.90:0/3352227618 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd21110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 -- 172.21.15.90:0/3352227618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 msgr2=0x556e1bd261e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd261e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 -- 172.21.15.90:0/3352227618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 msgr2=0x556e1bd28ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 -- 172.21.15.90:0/3352227618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1bc198b0 con 0x556e1bd20cf0 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a54e00640 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.842+0000 7f9a56200640 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd21110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a48011760 tx=0x7f9a48011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d938ed106bc76793 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.843+0000 7f9a42a00640 1 -- 172.21.15.90:0/3352227618 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48004d20 con 0x556e1bd20cf0 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.843+0000 7f9a4e000640 1 -- 172.21.15.90:0/1707942831 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9a440037a0 con 0x7f9a50104b10 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.843+0000 7f9a42a00640 1 -- 172.21.15.90:0/3352227618 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a48003970 con 0x556e1bd20cf0 2024-06-02T15:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.843+0000 7f9a42a00640 1 -- 172.21.15.90:0/3352227618 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48004380 con 0x556e1bd20cf0 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- 172.21.15.90:0/3352227618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 msgr2=0x556e1bd21110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd21110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9a48011760 tx=0x7f9a48011c20 comp rx=0 tx=0).stop 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- 172.21.15.90:0/3352227618 shutdown_connections 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/3352227618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd21110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- 172.21.15.90:0/3352227618 >> 172.21.15.90:0/3352227618 conn(0x556e1bd1c360 msgr2=0x556e1bd1c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- 172.21.15.90:0/3352227618 shutdown_connections 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- 172.21.15.90:0/3352227618 wait complete. 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a5b03d740 1 -- start start 2024-06-02T15:54:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.844+0000 7f9a4e000640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9a380b26f0 0x7f9a380b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a4e000640 1 -- 172.21.15.90:0/1707942831 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f9a440e2970 con 0x7f9a50104b10 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 Processor -- start 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a56200640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9a380b26f0 0x7f9a380b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd216e0 0x556e1bd2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556e1b966790 con 0x556e1bd216e0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x556e1bc28bc0 con 0x556e1bd20cf0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.845+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e1bd29800 con 0x556e1bd26850 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a5b03d740 10 client.0 _ll_register_callbacks cb 0x556e1bc06400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a54e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd216e0 0x556e1bd2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:54012/0 (socket says 172.21.15.90:54012) 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 -- 172.21.15.90:0/1796041158 learned_addr learned my addr 172.21.15.90:0/1796041158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a56200640 1 --2- 172.21.15.90:0/1707942831 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f9a380b26f0 0x7f9a380b4bb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a4801b040 tx=0x7f9a48005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a56200640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 -- 172.21.15.90:0/1796041158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 msgr2=0x556e1bd32110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd32110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 -- 172.21.15.90:0/1796041158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd216e0 msgr2=0x556e1bd2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd216e0 0x556e1bd2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.846+0000 7f9a55800640 1 -- 172.21.15.90:0/1796041158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1b995210 con 0x556e1bd20cf0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a55800640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9b70 tx=0x7f9a440a97b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc6ef5cbbff9332 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a37400640 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440adf90 con 0x556e1bd20cf0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f9a440a8e40 con 0x556e1bd20cf0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a48003970 con 0x556e1bd20cf0 2024-06-02T15:54:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9a440b2a40 con 0x556e1bd20cf0 2024-06-02T15:54:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a5b03d740 10 client.26540 fetch_fsmap learned FSMap version 40 2024-06-02T15:54:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556e1ba61620 con 0x556e1bd20cf0 2024-06-02T15:54:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.847+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a48003970 con 0x556e1bd20cf0 2024-06-02T15:54:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f9a440af120 con 0x556e1bd20cf0 2024-06-02T15:54:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 10 client.26540 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T15:54:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 10 client.26540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 20 client.26540 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bd36660 0x556e1bd38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 4 client.26540 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e1bc92ae0 con 0x556e1bd36660 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.848+0000 7f9a54e00640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bd36660 0x556e1bd38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.849+0000 7f9a54e00640 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bd36660 0x556e1bd38aa0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca63f573a79295e server_cookie=bc85c238f1101e0b in_seq=0 out_seq=0 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.849+0000 7f9a37400640 10 client.26540 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.849+0000 7f9a37400640 1 -- 172.21.15.90:0/1796041158 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e1bc92ae0 con 0x556e1bd36660 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.849+0000 7f9a37400640 10 client.26540 handle_command_reply: tid=0 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.849+0000 7f9a5b03d740 1 client.26540 shutdown 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 msgr2=0x556e1bd2abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a440a9b70 tx=0x7f9a440a97b0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 shutdown_connections 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bd36660 0x556e1bd38aa0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a37400640 0 client.26540 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd216e0 0x556e1bd2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/1796041158 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd2abe0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 >> 172.21.15.90:0/1796041158 conn(0x556e1bd1c360 msgr2=0x556e1bd1b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 shutdown_connections 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 1 -- 172.21.15.90:0/1796041158 wait complete. 2024-06-02T15:54:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.850+0000 7f9a5b03d740 20 client.26540 trim_cache size 0 max 16384 2024-06-02T15:54:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.867+0000 7f9a5b03d740 1 Processor -- start 2024-06-02T15:54:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 -- start start 2024-06-02T15:54:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd1f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd1ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501b0040 con 0x556e1bd26850 2024-06-02T15:54:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501ad850 con 0x556e1bd20cf0 2024-06-02T15:54:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501ab6a0 con 0x556e1bd216e0 2024-06-02T15:54:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a56200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd1ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a55800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd1f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a56200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd1ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44586/0 (socket says 172.21.15.90:44586) 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a56200640 1 -- 172.21.15.90:0/2813369716 learned_addr learned my addr 172.21.15.90:0/2813369716 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.868+0000 7f9a54e00640 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 -- 172.21.15.90:0/2813369716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 msgr2=0x556e1bd320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd320d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 -- 172.21.15.90:0/2813369716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 msgr2=0x556e1bd1ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd1ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 -- 172.21.15.90:0/2813369716 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1bd20930 con 0x556e1bd20cf0 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a55800640 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd1f980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9a440e0110 tx=0x7f9a440a9d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f7a8f4ed0d907f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a42000640 1 -- 172.21.15.90:0/2813369716 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440adf90 con 0x556e1bd20cf0 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a42000640 1 -- 172.21.15.90:0/2813369716 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a440c3060 con 0x556e1bd20cf0 2024-06-02T15:54:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.869+0000 7f9a42000640 1 -- 172.21.15.90:0/2813369716 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440a88b0 con 0x556e1bd20cf0 2024-06-02T15:54:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2813369716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 msgr2=0x556e1bd1f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd1f980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9a440e0110 tx=0x7f9a440a9d60 comp rx=0 tx=0).stop 2024-06-02T15:54:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2813369716 shutdown_connections 2024-06-02T15:54:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd26850 0x556e1bd1ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd216e0 0x556e1bd320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/2813369716 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd20cf0 0x556e1bd1f980 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2813369716 >> 172.21.15.90:0/2813369716 conn(0x556e1bd1c360 msgr2=0x556e1bd1f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2813369716 shutdown_connections 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2813369716 wait complete. 2024-06-02T15:54:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.870+0000 7f9a5b03d740 1 -- start start 2024-06-02T15:54:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 Processor -- start 2024-06-02T15:54:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd20cf0 0x556e1bd2f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd216e0 0x556e1bd2f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501abd10 con 0x556e1bd20cf0 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501ad180 con 0x556e1bd216e0 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a501ac3e0 con 0x556e1bd26850 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a5b03d740 10 client.0 _ll_register_callbacks cb 0x7f9a440046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a55800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd20cf0 0x556e1bd2f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a55800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd20cf0 0x556e1bd2f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44590/0 (socket says 172.21.15.90:44590) 2024-06-02T15:54:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a55800640 1 -- 172.21.15.90:0/2958908838 learned_addr learned my addr 172.21.15.90:0/2958908838 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a56200640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.871+0000 7f9a54e00640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd216e0 0x556e1bd2f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 -- 172.21.15.90:0/2958908838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 msgr2=0x556e1bd9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e1bd26850 0x556e1bd9f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 -- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd216e0 msgr2=0x556e1bd2f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x556e1bd216e0 0x556e1bd2f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 -- 172.21.15.90:0/2958908838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e1bd9f770 con 0x556e1bd20cf0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a55800640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556e1bd20cf0 0x556e1bd2f010 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9a440af4e0 tx=0x7f9a440a5960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f42650e02e9cef3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.872+0000 7f9a37400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a37400640 1 -- 172.21.15.90:0/2958908838 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440c9030 con 0x556e1bd20cf0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a37400640 1 -- 172.21.15.90:0/2958908838 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f9a440a61d0 con 0x556e1bd20cf0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2958908838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a440c3060 con 0x556e1bd20cf0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a37400640 1 -- 172.21.15.90:0/2958908838 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9a4408d050 con 0x556e1bd20cf0 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 10 client.16248 fetch_fsmap learned FSMap version 40 2024-06-02T15:54:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2958908838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556e1bda0990 con 0x556e1bd20cf0 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a37400640 1 -- 172.21.15.90:0/2958908838 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a440c3060 con 0x556e1bd20cf0 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a37400640 1 -- 172.21.15.90:0/2958908838 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f9a44085020 con 0x556e1bd20cf0 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 10 client.16248 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 20 client.16248 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.873+0000 7f9a5b03d740 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bda10f0 0x556e1bda34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.874+0000 7f9a5b03d740 4 client.16248 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T15:54:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.874+0000 7f9a5b03d740 1 -- 172.21.15.90:0/2958908838 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e1bc8c2c0 con 0x556e1bda10f0 2024-06-02T15:54:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.874+0000 7f9a54e00640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bda10f0 0x556e1bda34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.874+0000 7f9a54e00640 1 --2- 172.21.15.90:0/2958908838 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x556e1bda10f0 0x556e1bda34e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b03dbe028d7979 server_cookie=73f0792f91881566 in_seq=0 out_seq=0 2024-06-02T15:54:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-06-02T15:54:22.874+0000 7f9a37400640 10 client.16248 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:23.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:54:22.885+0000 7f0242966700 -1 log_channel(cluster) log [ERR] : failed to store backtrace on ino 0x10000005515 object, pool 2, errno -2 2024-06-02T15:54:23.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:54:22.885+0000 7f0242966700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-02T15:54:23.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:54:22.887+0000 7f0242966700 -1 log_channel(cluster) log [ERR] : failed to store backtrace on ino 0x10000005517 object, pool 2, errno -2 2024-06-02T15:54:23.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:54:22.887+0000 7f0242966700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-02T15:54:23.307 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:54:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T15:54:22.888+0000 7f0242966700 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-06-02T15:54:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:23 smithi090 ceph-mon[31595]: failed to store backtrace on ino 0x10000005515 object, pool 2, errno -2 2024-06-02T15:54:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:23 smithi090 ceph-mon[31595]: force file system read-only 2024-06-02T15:54:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:23 smithi090 ceph-mon[31595]: failed to store backtrace on ino 0x10000005517 object, pool 2, errno -2 2024-06-02T15:54:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 -- 172.21.15.90:0/4163408486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104630 msgr2=0x7f756c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 --2- 172.21.15.90:0/4163408486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104630 0x7f756c104a10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7560009990 tx=0x7f7560009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 -- 172.21.15.90:0/4163408486 shutdown_connections 2024-06-02T15:54:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 --2- 172.21.15.90:0/4163408486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c10da60 0x7f756c10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 --2- 172.21.15.90:0/4163408486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104630 0x7f756c104a10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 --2- 172.21.15.90:0/4163408486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c104f50 0x7f756c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 -- 172.21.15.90:0/4163408486 >> 172.21.15.90:0/4163408486 conn(0x7f756c0fde60 msgr2=0x7f756c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 -- 172.21.15.90:0/4163408486 shutdown_connections 2024-06-02T15:54:23.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.715+0000 7f7572400700 1 -- 172.21.15.90:0/4163408486 wait complete. 2024-06-02T15:54:23.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 Processor -- start 2024-06-02T15:54:23.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 -- start start 2024-06-02T15:54:23.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 0x7f756c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1136a0 con 0x7f756c10da60 2024-06-02T15:54:23.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f756c113520 con 0x7f756c104f50 2024-06-02T15:54:23.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.717+0000 7f7572400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f756c113820 con 0x7f756c104630 2024-06-02T15:54:23.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f756be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 0x7f756c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:56998/0 (socket says 172.21.15.90:56998) 2024-06-02T15:54:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f756be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:54038/0 (socket says 172.21.15.90:54038) 2024-06-02T15:54:23.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 -- 172.21.15.90:0/3159258111 learned_addr learned my addr 172.21.15.90:0/3159258111 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:23.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 -- 172.21.15.90:0/3159258111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 msgr2=0x7f756c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 -- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 msgr2=0x7f756c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 0x7f756c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 -- 172.21.15.90:0/3159258111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c19a180 con 0x7f756c104630 2024-06-02T15:54:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f756be00700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571a00700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 0x7f756c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:23.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.718+0000 7f7571000700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7560005910 tx=0x7f756000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.719+0000 7f7569600700 1 -- 172.21.15.90:0/3159258111 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560027070 con 0x7f756c104630 2024-06-02T15:54:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.719+0000 7f7569600700 1 -- 172.21.15.90:0/3159258111 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7560022470 con 0x7f756c104630 2024-06-02T15:54:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.719+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c19a410 con 0x7f756c104630 2024-06-02T15:54:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.719+0000 7f7569600700 1 -- 172.21.15.90:0/3159258111 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560017650 con 0x7f756c104630 2024-06-02T15:54:23.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.719+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a66e0 con 0x7f756c104630 2024-06-02T15:54:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.724+0000 7f7569600700 1 -- 172.21.15.90:0/3159258111 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f756002a070 con 0x7f756c104630 2024-06-02T15:54:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 Processor -- start 2024-06-02T15:54:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 -- start start 2024-06-02T15:54:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd35f0 0x5648fbfcd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfcd820 0x5648fbfcdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5648fbf26ec0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648fbf16a60 con 0x5648fbfd35f0 2024-06-02T15:54:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.725+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648fbf1aaa0 con 0x5648fbfcd820 2024-06-02T15:54:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44610/0 (socket says 172.21.15.90:44610) 2024-06-02T15:54:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 -- 172.21.15.90:0/2105400682 learned_addr learned my addr 172.21.15.90:0/2105400682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571000700 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd35f0 0x5648fbfcd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f756be00700 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfcd820 0x5648fbfcdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 -- 172.21.15.90:0/2105400682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfcd820 msgr2=0x5648fbfcdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfcd820 0x5648fbfcdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 -- 172.21.15.90:0/2105400682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd35f0 msgr2=0x5648fbfcd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd35f0 0x5648fbfcd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7571a00700 1 -- 172.21.15.90:0/2105400682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fbedf4a0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.726+0000 7f7569600700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 0x7f75540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f7569600700 1 -- 172.21.15.90:0/3159258111 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f75600e32b0 con 0x7f756c104630 2024-06-02T15:54:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f7571a00700 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7564003eb0 tx=0x7f7564009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8b80368ef056ba server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f756be00700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 0x7f75540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f755e000700 1 -- 172.21.15.90:0/2105400682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564019020 con 0x5648fbfd2cd0 2024-06-02T15:54:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f755e000700 1 -- 172.21.15.90:0/2105400682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75640092d0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.727+0000 7f755e000700 1 -- 172.21.15.90:0/2105400682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400e830 con 0x5648fbfd2cd0 2024-06-02T15:54:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f756be00700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 0x7f75540b4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5648fbfd28a0 tx=0x7f7558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 -- 172.21.15.90:0/2105400682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 msgr2=0x5648fbfd30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7564003eb0 tx=0x7f7564009920 comp rx=0 tx=0).stop 2024-06-02T15:54:23.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 -- 172.21.15.90:0/2105400682 shutdown_connections 2024-06-02T15:54:23.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfcd820 0x5648fbfcdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd30b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/2105400682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd35f0 0x5648fbfcd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 -- 172.21.15.90:0/2105400682 >> 172.21.15.90:0/2105400682 conn(0x5648fbef6c60 msgr2=0x5648fbfc3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.728+0000 7f757e2c1b80 1 -- 172.21.15.90:0/2105400682 shutdown_connections 2024-06-02T15:54:23.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.729+0000 7f757e2c1b80 1 -- 172.21.15.90:0/2105400682 wait complete. 2024-06-02T15:54:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.729+0000 7f757e2c1b80 1 -- start start 2024-06-02T15:54:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.729+0000 7f757e2c1b80 1 Processor -- start 2024-06-02T15:54:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd6ba0 0x5648fbfd7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5648fbe1d020 con 0x5648fbfd6ba0 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5648fbeb77e0 con 0x5648fbfcd820 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648fbfda090 con 0x5648fbfd2cd0 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f757e2c1b80 10 client.0 _ll_register_callbacks cb 0x5648fbec0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f7571000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f7571a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd6ba0 0x5648fbfd7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f7571000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:54052/0 (socket says 172.21.15.90:54052) 2024-06-02T15:54:23.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f7571000700 1 -- 172.21.15.90:0/1714193546 learned_addr learned my addr 172.21.15.90:0/1714193546 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:23.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.730+0000 7f756be00700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 -- 172.21.15.90:0/1714193546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 msgr2=0x5648fbfd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 -- 172.21.15.90:0/1714193546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd6ba0 msgr2=0x5648fbfd7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd6ba0 0x5648fbfd7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 -- 172.21.15.90:0/1714193546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fbe66950 con 0x5648fbfcd820 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7571000700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75600a65d0 tx=0x7f7560005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17551c9acc3def64 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7552a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:23.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75600e0e60 con 0x5648fbfcd820 2024-06-02T15:54:23.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75640092d0 con 0x5648fbfcd820 2024-06-02T15:54:23.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.731+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f75600aa7b0 con 0x5648fbfcd820 2024-06-02T15:54:23.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f75600a6410 con 0x5648fbfcd820 2024-06-02T15:54:23.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 10 client.26558 fetch_fsmap learned FSMap version 40 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5648fbee70e0 con 0x5648fbfcd820 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75640092d0 con 0x5648fbfcd820 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f75600b2e90 con 0x5648fbfcd820 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 10 client.26558 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 10 client.26558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 20 client.26558 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfe7090 0x5648fbfe94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 4 client.26558 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.732+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648fbf28e10 con 0x5648fbfe7090 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.733+0000 7f756be00700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfe7090 0x5648fbfe94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.733+0000 7f756be00700 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfe7090 0x5648fbfe94d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cb13e26fe2b601 server_cookie=39c027bb05e0eb97 in_seq=0 out_seq=0 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.733+0000 7f7552a00700 10 client.26558 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f7552a00700 1 -- 172.21.15.90:0/1714193546 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5648fbf28e10 con 0x5648fbfe7090 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f7552a00700 10 client.26558 handle_command_reply: tid=0 2024-06-02T15:54:23.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 client.26558 shutdown 2024-06-02T15:54:23.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 msgr2=0x5648fbfd92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f75600a65d0 tx=0x7f7560005e80 comp rx=0 tx=0).stop 2024-06-02T15:54:23.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 shutdown_connections 2024-06-02T15:54:23.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfe7090 0x5648fbfe94d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfd6ba0 0x5648fbfd7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfcd820 0x5648fbfd92c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1714193546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f7552a00700 0 client.26558 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:23.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.734+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 >> 172.21.15.90:0/1714193546 conn(0x5648fbef6c60 msgr2=0x5648fbfc3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.735+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 shutdown_connections 2024-06-02T15:54:23.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.735+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1714193546 wait complete. 2024-06-02T15:54:23.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.735+0000 7f757e2c1b80 20 client.26558 trim_cache size 0 max 16384 2024-06-02T15:54:23.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.763+0000 7f757e2c1b80 1 Processor -- start 2024-06-02T15:54:23.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 -- start start 2024-06-02T15:54:23.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfc84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1af890 con 0x5648fbfcd820 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ad060 con 0x5648fbfd6ba0 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1aaed0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f7571000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f756be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.764+0000 7f7571000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44620/0 (socket says 172.21.15.90:44620) 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f7571000700 1 -- 172.21.15.90:0/1733249788 learned_addr learned my addr 172.21.15.90:0/1733249788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:57012/0 (socket says 172.21.15.90:57012) 2024-06-02T15:54:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f7571a00700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfc84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 -- 172.21.15.90:0/1733249788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 msgr2=0x5648fbfc84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfc84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 -- 172.21.15.90:0/1733249788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 msgr2=0x5648fbfd57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 -- 172.21.15.90:0/1733249788 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fbfd98a0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f7571a00700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfc84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f7571000700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.765+0000 7f756be00700 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7558009fd0 tx=0x7f755801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8990000073dc0b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.766+0000 7f755d600700 1 -- 172.21.15.90:0/1733249788 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755801ecd0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.766+0000 7f755d600700 1 -- 172.21.15.90:0/1733249788 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755802a8d0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.766+0000 7f755d600700 1 -- 172.21.15.90:0/1733249788 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755801d960 con 0x5648fbfd2cd0 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1733249788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 msgr2=0x5648fbfc37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7558009fd0 tx=0x7f755801a000 comp rx=0 tx=0).stop 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1733249788 shutdown_connections 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfc84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfd57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/1733249788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfc37c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1733249788 >> 172.21.15.90:0/1733249788 conn(0x5648fbef6c60 msgr2=0x5648fbfc7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1733249788 shutdown_connections 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.767+0000 7f757e2c1b80 1 -- 172.21.15.90:0/1733249788 wait complete. 2024-06-02T15:54:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 -- start start 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 Processor -- start 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfdcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfdd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ab520 con 0x5648fbfcd820 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ac990 con 0x5648fbfd6ba0 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.768+0000 7f757e2c1b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1abbf0 con 0x5648fbfd2cd0 2024-06-02T15:54:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f757e2c1b80 10 client.0 _ll_register_callbacks cb 0x5648fbec0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f756be00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfdcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44636/0 (socket says 172.21.15.90:44636) 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 -- 172.21.15.90:0/417802590 learned_addr learned my addr 172.21.15.90:0/417802590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571a00700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfdd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 -- 172.21.15.90:0/417802590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 msgr2=0x5648fbfdcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfdcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 -- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 msgr2=0x5648fbfdd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfdd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571000700 1 -- 172.21.15.90:0/417802590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fbfdd540 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.769+0000 7f7571a00700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfdd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f7571000700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7560005b40 tx=0x7f7560022c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb0c9b5605ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f7552a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75600e0e60 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755802a8d0 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f75600a6b40 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f75600c4050 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f757e2c1b80 10 client.16266 fetch_fsmap learned FSMap version 40 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.770+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5648fbfc8df0 con 0x5648fbfcd820 2024-06-02T15:54:23.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755802a8d0 con 0x5648fbfcd820 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f756000fac0 con 0x5648fbfcd820 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 10 client.16266 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 20 client.16266 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfc9310 0x5648fbfcb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 4 client.16266 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648fbf225f0 con 0x5648fbfc9310 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.771+0000 7f756be00700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfc9310 0x5648fbfcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.772+0000 7f756be00700 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfc9310 0x5648fbfcb700 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9ba002dbe12e49 server_cookie=a19afad9052c40af in_seq=0 out_seq=0 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.772+0000 7f7552a00700 10 client.16266 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f7552a00700 1 -- 172.21.15.90:0/417802590 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648fbf225f0 con 0x5648fbfc9310 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f7552a00700 10 client.16266 handle_command_reply: tid=0 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 client.16266 shutdown 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 msgr2=0x5648fbfdf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7560005b40 tx=0x7f7560022c00 comp rx=0 tx=0).stop 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 shutdown_connections 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5648fbfc9310 0x5648fbfcb700 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5648fbfd6ba0 0x5648fbfdd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5648fbfcd820 0x5648fbfdf720 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 --2- 172.21.15.90:0/417802590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648fbfd2cd0 0x5648fbfdcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 >> 172.21.15.90:0/417802590 conn(0x5648fbef6c60 msgr2=0x5648fbfd1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f7552a00700 0 client.16266 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 shutdown_connections 2024-06-02T15:54:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 1 -- 172.21.15.90:0/417802590 wait complete. 2024-06-02T15:54:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.773+0000 7f757e2c1b80 20 client.16266 trim_cache size 0 max 16384 2024-06-02T15:54:23.775 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:54:23.775 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:54:23.775 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "06fda705-5a2a-422d-978c-e0cddf766e50", 2024-06-02T15:54:23.775 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:54:23.775 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:54:23.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.774+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 msgr2=0x7f75540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.774+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 0x7f75540b4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5648fbfd28a0 tx=0x7f7558009260 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.774+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 msgr2=0x7f756c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.774+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7560005910 tx=0x7f756000b670 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 shutdown_connections 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f75540b2900 0x7f75540b4dc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f756c10da60 0x7f756c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f756c104630 0x7f756c19f810 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 --2- 172.21.15.90:0/3159258111 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f756c104f50 0x7f756c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 >> 172.21.15.90:0/3159258111 conn(0x7f756c0fde60 msgr2=0x7f756c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 shutdown_connections 2024-06-02T15:54:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:23.775+0000 7f7572400700 1 -- 172.21.15.90:0/3159258111 wait complete. 2024-06-02T15:54:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:23 smithi120 ceph-mon[36451]: failed to store backtrace on ino 0x10000005515 object, pool 2, errno -2 2024-06-02T15:54:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:23 smithi120 ceph-mon[36451]: force file system read-only 2024-06-02T15:54:23.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:23 smithi120 ceph-mon[36451]: failed to store backtrace on ino 0x10000005517 object, pool 2, errno -2 2024-06-02T15:54:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:23 smithi135 ceph-mon[36839]: failed to store backtrace on ino 0x10000005515 object, pool 2, errno -2 2024-06-02T15:54:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:23 smithi135 ceph-mon[36839]: force file system read-only 2024-06-02T15:54:23.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:23 smithi135 ceph-mon[36839]: failed to store backtrace on ino 0x10000005517 object, pool 2, errno -2 2024-06-02T15:54:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:24 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:54:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:24 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:24 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:54:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:24 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:24 smithi090 ceph-mon[31595]: pgmap v817: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-06-02T15:54:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:24 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:54:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:24 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:24 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:54:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:24 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:24 smithi120 ceph-mon[36451]: pgmap v817: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-06-02T15:54:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:24 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:54:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:24 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:24 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:54:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:24 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:24 smithi135 ceph-mon[36839]: pgmap v817: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-06-02T15:54:25.009 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:54:25.516 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:27.510 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:27 smithi120 ceph-mon[36451]: pgmap v818: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 24 op/s 2024-06-02T15:54:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:27 smithi135 ceph-mon[36839]: pgmap v818: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 24 op/s 2024-06-02T15:54:27.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:27 smithi090 ceph-mon[31595]: pgmap v818: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 24 op/s 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 -- 172.21.15.90:0/3613460960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 msgr2=0x7fb72c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 --2- 172.21.15.90:0/3613460960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c10fa00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb718009990 tx=0x7fb718009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 -- 172.21.15.90:0/3613460960 shutdown_connections 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 --2- 172.21.15.90:0/3613460960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 0x7fb72c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 --2- 172.21.15.90:0/3613460960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 --2- 172.21.15.90:0/3613460960 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c10fa00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 -- 172.21.15.90:0/3613460960 >> 172.21.15.90:0/3613460960 conn(0x7fb72c07a6a0 msgr2=0x7fb72c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 -- 172.21.15.90:0/3613460960 shutdown_connections 2024-06-02T15:54:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.298+0000 7fb733a00700 1 -- 172.21.15.90:0/3613460960 wait complete. 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.299+0000 7fb733a00700 1 Processor -- start 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.299+0000 7fb733a00700 1 -- start start 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 0x7fb72c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115950 con 0x7fb72c075770 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1157d0 con 0x7fb72c076090 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb733a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c115ad0 con 0x7fb72c10ff40 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb72be00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c19be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44652/0 (socket says 172.21.15.90:44652) 2024-06-02T15:54:28.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb731000700 1 -- 172.21.15.90:0/3926142434 learned_addr learned my addr 172.21.15.90:0/3926142434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:28.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.300+0000 7fb731a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 0x7fb72c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 -- 172.21.15.90:0/3926142434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 msgr2=0x7fb72c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 0x7fb72c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 -- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 msgr2=0x7fb72c19be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c19be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 -- 172.21.15.90:0/3926142434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1a0e30 con 0x7fb72c076090 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb72be00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c19c360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7180098e0 tx=0x7fb7180057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb729600700 1 -- 172.21.15.90:0/3926142434 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718027070 con 0x7fb72c076090 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1a10c0 con 0x7fb72c076090 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb729600700 1 -- 172.21.15.90:0/3926142434 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb718022470 con 0x7fb72c076090 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.301+0000 7fb729600700 1 -- 172.21.15.90:0/3926142434 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718017650 con 0x7fb72c076090 2024-06-02T15:54:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.302+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1a89a0 con 0x7fb72c076090 2024-06-02T15:54:28.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.305+0000 7fb729600700 1 -- 172.21.15.90:0/3926142434 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb71802a070 con 0x7fb72c076090 2024-06-02T15:54:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.306+0000 7fb73e2c8b80 1 Processor -- start 2024-06-02T15:54:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.306+0000 7fb729600700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 0x7fb7140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.306+0000 7fb73e2c8b80 1 -- start start 2024-06-02T15:54:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb729600700 1 -- 172.21.15.90:0/3926142434 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb7180e3290 con 0x7fb72c076090 2024-06-02T15:54:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb731a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 0x7fb7140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008f30 0x55e9ad002b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad0035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e9acf60e40 con 0x55e9ad008f30 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9acf509e0 con 0x55e9ad008610 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e9acf54a20 con 0x55e9ad003140 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb731000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb731000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:54100/0 (socket says 172.21.15.90:54100) 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb731000700 1 -- 172.21.15.90:0/3766393803 learned_addr learned my addr 172.21.15.90:0/3766393803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb72be00700 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008f30 0x55e9ad002b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.307+0000 7fb731a00700 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad0035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 -- 172.21.15.90:0/3766393803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 msgr2=0x55e9ad0035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad0035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 -- 172.21.15.90:0/3766393803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008f30 msgr2=0x55e9ad002b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008f30 0x55e9ad002b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 -- 172.21.15.90:0/3766393803 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9acf19420 con 0x55e9ad008610 2024-06-02T15:54:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731000700 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e660 tx=0x7fb720009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415102a455c5f900 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb731a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 0x7fb7140b4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb72c06bdd0 tx=0x7fb7240118e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb71e000700 1 -- 172.21.15.90:0/3766393803 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720003a00 con 0x55e9ad008610 2024-06-02T15:54:28.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb71e000700 1 -- 172.21.15.90:0/3766393803 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb72001a070 con 0x55e9ad008610 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.308+0000 7fb71e000700 1 -- 172.21.15.90:0/3766393803 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720007760 con 0x55e9ad008610 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3766393803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 msgr2=0x55e9ad0089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e660 tx=0x7fb720009920 comp rx=0 tx=0).stop 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3766393803 shutdown_connections 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad0035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad0089f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3766393803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008f30 0x55e9ad002b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3766393803 >> 172.21.15.90:0/3766393803 conn(0x55e9acf30be0 msgr2=0x55e9acffd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3766393803 shutdown_connections 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.309+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3766393803 wait complete. 2024-06-02T15:54:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.310+0000 7fb73e2c8b80 1 -- start start 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.310+0000 7fb73e2c8b80 1 Processor -- start 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad00c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008f30 0x55e9ad00c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e9ace56fa0 con 0x55e9ad003140 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e9acef1760 con 0x55e9ad008610 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e9ad00f980 con 0x55e9ad008f30 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb73e2c8b80 10 client.0 _ll_register_callbacks cb 0x55e9acefac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008f30 0x55e9ad00c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44682/0 (socket says 172.21.15.90:44682) 2024-06-02T15:54:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 -- 172.21.15.90:0/1637189006 learned_addr learned my addr 172.21.15.90:0/1637189006 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb72be00700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 -- 172.21.15.90:0/1637189006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008f30 msgr2=0x55e9ad00c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008f30 0x55e9ad00c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 -- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 msgr2=0x55e9ad00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad00c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.311+0000 7fb731000700 1 -- 172.21.15.90:0/1637189006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9acea08d0 con 0x55e9ad003140 2024-06-02T15:54:28.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb731000700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e250 tx=0x7fb720003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364dc4827e0a5039 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb712a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720004050 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb7200041f0 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72001a070 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.312+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb720004410 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 10 client.16278 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e9acf21060 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb72001a070 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb7200155c0 con 0x55e9ad003140 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 20 client.16278 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad01c520 0x55e9ad01e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 4 client.16278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.313+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9acf62d90 con 0x55e9ad01c520 2024-06-02T15:54:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.314+0000 7fb72be00700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad01c520 0x55e9ad01e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.314+0000 7fb72be00700 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad01c520 0x55e9ad01e910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e5d601daf55951 server_cookie=8b024343d71597ca in_seq=0 out_seq=0 2024-06-02T15:54:28.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.314+0000 7fb712a00700 10 client.16278 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb712a00700 1 -- 172.21.15.90:0/1637189006 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9acf62d90 con 0x55e9ad01c520 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb712a00700 10 client.16278 handle_command_reply: tid=0 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 client.16278 shutdown 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 msgr2=0x55e9ad00ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb72000e250 tx=0x7fb720003a00 comp rx=0 tx=0).stop 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 shutdown_connections 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad01c520 0x55e9ad01e910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008f30 0x55e9ad00c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad00ed30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/1637189006 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008610 0x55e9ad00c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 >> 172.21.15.90:0/1637189006 conn(0x55e9acf30be0 msgr2=0x55e9acffcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.315+0000 7fb712a00700 0 client.16278 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.316+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 shutdown_connections 2024-06-02T15:54:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.316+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/1637189006 wait complete. 2024-06-02T15:54:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.316+0000 7fb73e2c8b80 20 client.16278 trim_cache size 0 max 16384 2024-06-02T15:54:28.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.338+0000 7fb73e2c8b80 1 Processor -- start 2024-06-02T15:54:28.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.338+0000 7fb73e2c8b80 1 -- start start 2024-06-02T15:54:28.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad002b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9acffd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1b3e30 con 0x55e9ad008610 2024-06-02T15:54:28.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1b1600 con 0x55e9ad008f30 2024-06-02T15:54:28.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1af450 con 0x55e9ad003140 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb72be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb72be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44686/0 (socket says 172.21.15.90:44686) 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb72be00700 1 -- 172.21.15.90:0/3949575175 learned_addr learned my addr 172.21.15.90:0/3949575175 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb731000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad002b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.339+0000 7fb731a00700 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9acffd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 -- 172.21.15.90:0/3949575175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 msgr2=0x55e9ad002b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad002b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 -- 172.21.15.90:0/3949575175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 msgr2=0x55e9acffd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9acffd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 -- 172.21.15.90:0/3949575175 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ad008c40 con 0x55e9ad008610 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb72be00700 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7180a65b0 tx=0x7fb7180e0850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7c1a24d141e536 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb71d600700 1 -- 172.21.15.90:0/3949575175 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180ae8e0 con 0x55e9ad008610 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb71d600700 1 -- 172.21.15.90:0/3949575175 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7180b3300 con 0x55e9ad008610 2024-06-02T15:54:28.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.340+0000 7fb71d600700 1 -- 172.21.15.90:0/3949575175 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180aa790 con 0x55e9ad008610 2024-06-02T15:54:28.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3949575175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 msgr2=0x55e9ad00c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb7180a65b0 tx=0x7fb7180e0850 comp rx=0 tx=0).stop 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3949575175 shutdown_connections 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9acffd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad003140 0x55e9ad002b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/3949575175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad008610 0x55e9ad00c520 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3949575175 >> 172.21.15.90:0/3949575175 conn(0x55e9acf30be0 msgr2=0x55e9ad009ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3949575175 shutdown_connections 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.341+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/3949575175 wait complete. 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.342+0000 7fb73e2c8b80 1 -- start start 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.342+0000 7fb73e2c8b80 1 Processor -- start 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.342+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008610 0x55e9ad015330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9ad011be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1afac0 con 0x55e9ad003140 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1b0f30 con 0x55e9ad008f30 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c1b0190 con 0x55e9ad008610 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb73e2c8b80 10 client.0 _ll_register_callbacks cb 0x55e9acefac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44690/0 (socket says 172.21.15.90:44690) 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 -- 172.21.15.90:0/4176697293 learned_addr learned my addr 172.21.15.90:0/4176697293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb72be00700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008610 0x55e9ad015330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731a00700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9ad011be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 -- 172.21.15.90:0/4176697293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008610 msgr2=0x55e9ad015330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008610 0x55e9ad015330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 -- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 msgr2=0x55e9ad011be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9ad011be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731000700 1 -- 172.21.15.90:0/4176697293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ad012330 con 0x55e9ad003140 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.343+0000 7fb731a00700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9ad011be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.344+0000 7fb731000700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb720015b80 tx=0x7fb720015e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b1d68afa34a36 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.344+0000 7fb712a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.344+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720004200 con 0x55e9ad003140 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.344+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb720015450 con 0x55e9ad003140 2024-06-02T15:54:28.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.344+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7180b3300 con 0x55e9ad003140 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.345+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb72003c050 con 0x55e9ad003140 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.345+0000 7fb73e2c8b80 10 client.16284 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.345+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e9ad087a10 con 0x55e9ad003140 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7180b3300 con 0x55e9ad003140 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fb720004cb0 con 0x55e9ad003140 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 20 client.16284 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad088170 0x55e9ad08a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 4 client.16284 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9acf5c570 con 0x55e9ad088170 2024-06-02T15:54:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.346+0000 7fb72be00700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad088170 0x55e9ad08a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:28.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb72be00700 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad088170 0x55e9ad08a590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d3d97f2d89d6d8 server_cookie=eb44bcfdae981d3d in_seq=0 out_seq=0 2024-06-02T15:54:28.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb712a00700 10 client.16284 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:28.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb712a00700 1 -- 172.21.15.90:0/4176697293 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9acf5c570 con 0x55e9ad088170 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb712a00700 10 client.16284 handle_command_reply: tid=0 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 client.16284 shutdown 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 msgr2=0x55e9ad014df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb720015b80 tx=0x7fb720015e30 comp rx=0 tx=0).stop 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 shutdown_connections 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e9ad088170 0x55e9ad08a590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e9ad008f30 0x55e9ad011be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9ad003140 0x55e9ad014df0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 --2- 172.21.15.90:0/4176697293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9ad008610 0x55e9ad015330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 >> 172.21.15.90:0/4176697293 conn(0x55e9acf30be0 msgr2=0x55e9ad009ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.347+0000 7fb712a00700 0 client.16284 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 shutdown_connections 2024-06-02T15:54:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb73e2c8b80 1 -- 172.21.15.90:0/4176697293 wait complete. 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb73e2c8b80 20 client.16284 trim_cache size 0 max 16384 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:54:28.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 msgr2=0x7fb7140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 0x7fb7140b4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb72c06bdd0 tx=0x7fb7240118e0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 msgr2=0x7fb72c19c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.348+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c19c360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7180098e0 tx=0x7fb7180057c0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 shutdown_connections 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb7140b2880 0x7fb7140b4d40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c10ff40 0x7fb72c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb72c075770 0x7fb72c19be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 --2- 172.21.15.90:0/3926142434 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb72c076090 0x7fb72c19c360 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 >> 172.21.15.90:0/3926142434 conn(0x7fb72c07a6a0 msgr2=0x7fb72c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 shutdown_connections 2024-06-02T15:54:28.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:28.349+0000 7fb733a00700 1 -- 172.21.15.90:0/3926142434 wait complete. 2024-06-02T15:54:28.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:28 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:54:28.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:28 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:54:28.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:28 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:54:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:29 smithi120 ceph-mon[36451]: pgmap v819: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-02T15:54:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:29 smithi120 ceph-mon[36451]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-02T15:54:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:29 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:54:29.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:29 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:29 smithi135 ceph-mon[36839]: pgmap v819: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-02T15:54:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:29 smithi135 ceph-mon[36839]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-02T15:54:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:29 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:54:29.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:29 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:29.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:29 smithi090 ceph-mon[31595]: pgmap v819: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-06-02T15:54:29.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:29 smithi090 ceph-mon[31595]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-06-02T15:54:29.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:29 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] up:active 2024-06-02T15:54:29.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:29 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:54:29.671 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:54:29.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:54:30.133 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:31 smithi120 ceph-mon[36451]: pgmap v820: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-06-02T15:54:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:31 smithi135 ceph-mon[36839]: pgmap v820: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-06-02T15:54:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:31 smithi090 ceph-mon[31595]: pgmap v820: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 -- 172.21.15.90:0/695227609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8106120 msgr2=0x7f77a8112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 --2- 172.21.15.90:0/695227609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8106120 0x7f77a8112630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f77a000b0a0 tx=0x7f77a000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 -- 172.21.15.90:0/695227609 shutdown_connections 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 --2- 172.21.15.90:0/695227609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8106120 0x7f77a8112630 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 --2- 172.21.15.90:0/695227609 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 0x7f77a810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 --2- 172.21.15.90:0/695227609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8105760 0x7f77a8105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 -- 172.21.15.90:0/695227609 >> 172.21.15.90:0/695227609 conn(0x7f77a8101250 msgr2=0x7f77a8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 -- 172.21.15.90:0/695227609 shutdown_connections 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.024+0000 7f77af200700 1 -- 172.21.15.90:0/695227609 wait complete. 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.025+0000 7f77af200700 1 Processor -- start 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.025+0000 7f77af200700 1 -- start start 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8106120 0x7f77a819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 0x7f77a81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81159b0 con 0x7f77a8105760 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8115830 con 0x7f77a810b760 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77af200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8115b30 con 0x7f77a8106120 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77a7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37942/0 (socket says 172.21.15.90:37942) 2024-06-02T15:54:33.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77a7e00700 1 -- 172.21.15.90:0/1592836427 learned_addr learned my addr 172.21.15.90:0/1592836427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:33.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f779fe00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8106120 0x7f77a819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:33.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.026+0000 7f77ad200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 0x7f77a81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:33.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 -- 172.21.15.90:0/1592836427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8106120 msgr2=0x7f77a819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8106120 0x7f77a819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 -- 172.21.15.90:0/1592836427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 msgr2=0x7f77a81a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 0x7f77a81a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 -- 172.21.15.90:0/1592836427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a81a0e10 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a7e00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f779800d980 tx=0x7f779800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798016070 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7798004830 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a81a1100 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.027+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798005540 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.028+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81a8980 con 0x7f77a8105760 2024-06-02T15:54:33.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.029+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a806a230 con 0x7f77a8105760 2024-06-02T15:54:33.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.031+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7798004220 con 0x7f77a8105760 2024-06-02T15:54:33.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.032+0000 7f77a5600700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 0x7f778c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:33.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.033+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f77980e2820 con 0x7f77a8105760 2024-06-02T15:54:33.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.033+0000 7f779fe00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 0x7f778c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:33.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.033+0000 7f779fe00700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 0x7f778c0b4e40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f77a8103820 tx=0x7f7790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:33.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.035+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f779801f020 con 0x7f77a8105760 2024-06-02T15:54:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.315+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77a802e560 con 0x7f77a8105760 2024-06-02T15:54:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.318+0000 7f77a5600700 1 -- 172.21.15.90:0/1592836427 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11889 (secure 0 0 0) 0x7f77980aa130 con 0x7f77a8105760 2024-06-02T15:54:33.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:54:33.315 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:54:27.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:54:33.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.321+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 msgr2=0x7f778c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.321+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 0x7f778c0b4e40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f77a8103820 tx=0x7f7790009260 comp rx=0 tx=0).stop 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 msgr2=0x7f77a819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f779800d980 tx=0x7f779800dc90 comp rx=0 tx=0).stop 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 shutdown_connections 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f778c0b2980 0x7f778c0b4e40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f77a810b760 0x7f77a81a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77a8105760 0x7f77a819be00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 --2- 172.21.15.90:0/1592836427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a8106120 0x7f77a819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:33.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 >> 172.21.15.90:0/1592836427 conn(0x7f77a8101250 msgr2=0x7f77a81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:33.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 shutdown_connections 2024-06-02T15:54:33.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:33.322+0000 7f77af200700 1 -- 172.21.15.90:0/1592836427 wait complete. 2024-06-02T15:54:33.320 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-06-02T15:54:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:33 smithi120 ceph-mon[36451]: pgmap v821: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-06-02T15:54:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:33 smithi135 ceph-mon[36839]: pgmap v821: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-06-02T15:54:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:33 smithi090 ceph-mon[31595]: pgmap v821: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-06-02T15:54:34.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:54:34.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:34 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1592836427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:34.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:34 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1592836427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:34.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:34 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1592836427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:54:34.643 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:35.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:35 smithi090 ceph-mon[31595]: pgmap v822: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-06-02T15:54:35.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:35 smithi120 ceph-mon[36451]: pgmap v822: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-06-02T15:54:35.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:35 smithi135 ceph-mon[36839]: pgmap v822: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-06-02T15:54:36.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:36 smithi120 ceph-mon[36451]: pgmap v823: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-06-02T15:54:36.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:36 smithi135 ceph-mon[36839]: pgmap v823: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-06-02T15:54:36.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:36 smithi090 ceph-mon[31595]: pgmap v823: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-06-02T15:54:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 -- 172.21.15.90:0/1660904072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 msgr2=0x7fbdec075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1660904072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec075b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009990 tx=0x7fbde0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 -- 172.21.15.90:0/1660904072 shutdown_connections 2024-06-02T15:54:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1660904072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec114710 0x7fbdec116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1660904072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec075b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1660904072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec076090 0x7fbdec1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 -- 172.21.15.90:0/1660904072 >> 172.21.15.90:0/1660904072 conn(0x7fbdec07a6a0 msgr2=0x7fbdec07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 -- 172.21.15.90:0/1660904072 shutdown_connections 2024-06-02T15:54:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.976+0000 7fbdf3800700 1 -- 172.21.15.90:0/1660904072 wait complete. 2024-06-02T15:54:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.977+0000 7fbdf3800700 1 Processor -- start 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.977+0000 7fbdf3800700 1 -- start start 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec114710 0x7fbdec1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec119d00 con 0x7fbdec075770 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec119b80 con 0x7fbdec076090 2024-06-02T15:54:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf3800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec119e80 con 0x7fbdec114710 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35194/0 (socket says 172.21.15.90:35194) 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdebe00700 1 -- 172.21.15.90:0/1986359532 learned_addr learned my addr 172.21.15.90:0/1986359532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.978+0000 7fbdf1800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec114710 0x7fbdec1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 -- 172.21.15.90:0/1986359532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec114710 msgr2=0x7fbdec1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec114710 0x7fbdec1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 -- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 msgr2=0x7fbdec1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 -- 172.21.15.90:0/1986359532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1a5240 con 0x7fbdec076090 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdebe00700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e9a0 tx=0x7fbdd800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbde9600700 1 -- 172.21.15.90:0/1986359532 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800c9c0 con 0x7fbdec076090 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbde9600700 1 -- 172.21.15.90:0/1986359532 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd800cb60 con 0x7fbdec076090 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1a5530 con 0x7fbdec076090 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.979+0000 7fbde9600700 1 -- 172.21.15.90:0/1986359532 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8018680 con 0x7fbdec076090 2024-06-02T15:54:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.980+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec1acdb0 con 0x7fbdec076090 2024-06-02T15:54:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.983+0000 7fbde9600700 1 -- 172.21.15.90:0/1986359532 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbdd8010430 con 0x7fbdec076090 2024-06-02T15:54:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.985+0000 7fbdfe266b80 1 Processor -- start 2024-06-02T15:54:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.985+0000 7fbdfe266b80 1 -- start start 2024-06-02T15:54:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fbeb0 0x5621aa1fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 0x5621aa1f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5621aa154e40 con 0x5621aa1f6880 2024-06-02T15:54:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621aa1449e0 con 0x5621aa1fc7d0 2024-06-02T15:54:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621aa148a20 con 0x5621aa1fbeb0 2024-06-02T15:54:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbde9600700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 0x7fbdd40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35210/0 (socket says 172.21.15.90:35210) 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2787489163 learned_addr learned my addr 172.21.15.90:0/2787489163 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf1800700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fbeb0 0x5621aa1fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbde9600700 1 -- 172.21.15.90:0/1986359532 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fbdd80e28b0 con 0x7fbdec076090 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf1800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 0x7fbdd40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdebe00700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 0x5621aa1f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2787489163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fbeb0 msgr2=0x5621aa1fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fbeb0 0x5621aa1fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2787489163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 msgr2=0x5621aa1f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.986+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 0x5621aa1f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2787489163 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621aa10d420 con 0x5621aa1fc7d0 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdebe00700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 0x5621aa1f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:54:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009990 tx=0x7fbde00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483f262d32cb60b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdde000700 1 -- 172.21.15.90:0/2787489163 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0027070 con 0x5621aa1fc7d0 2024-06-02T15:54:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdde000700 1 -- 172.21.15.90:0/2787489163 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde00056a0 con 0x5621aa1fc7d0 2024-06-02T15:54:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.987+0000 7fbdde000700 1 -- 172.21.15.90:0/2787489163 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0003a70 con 0x5621aa1fc7d0 2024-06-02T15:54:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2787489163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 msgr2=0x5621aa1f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdf1800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 0x7fbdd40b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009da0 tx=0x7fbde400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009990 tx=0x7fbde00096a0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2787489163 shutdown_connections 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1f6880 0x5621aa1f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fbeb0 0x5621aa1fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2787489163 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f6260 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.988+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2787489163 >> 172.21.15.90:0/2787489163 conn(0x5621aa124be0 msgr2=0x5621aa1f1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.989+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2787489163 shutdown_connections 2024-06-02T15:54:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.989+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2787489163 wait complete. 2024-06-02T15:54:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.989+0000 7fbdfe266b80 1 -- start start 2024-06-02T15:54:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.989+0000 7fbdfe266b80 1 Processor -- start 2024-06-02T15:54:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa1ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa1ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5621aa04afa0 con 0x5621aa1fbeb0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5621aa0e5760 con 0x5621aa1f6880 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621aa2031e0 con 0x5621aa1fc7d0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdfe266b80 10 client.0 _ll_register_callbacks cb 0x5621aa0eec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35218/0 (socket says 172.21.15.90:35218) 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2058361379 learned_addr learned my addr 172.21.15.90:0/2058361379 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdf1800700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa1ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.990+0000 7fbdebe00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa1ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2058361379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 msgr2=0x5621aa1ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa1ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 msgr2=0x5621aa1ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa1ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 -- 172.21.15.90:0/2058361379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621aa0948d0 con 0x5621aa1f6880 2024-06-02T15:54:37.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf1800700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa1ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009350 tx=0x7fbde000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a163de5cc1ba76e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0033060 con 0x5621aa1f6880 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde00056a0 con 0x5621aa1f6880 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.991+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fbde0003890 con 0x5621aa1f6880 2024-06-02T15:54:37.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbde0027070 con 0x5621aa1f6880 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 10 client.26600 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5621aa115060 con 0x5621aa1f6880 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde00056a0 con 0x5621aa1f6880 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fbde0021070 con 0x5621aa1f6880 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 10 client.26600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 20 client.26600 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa20fad0 0x5621aa211ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 4 client.26600 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.992+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621aa156d90 con 0x5621aa20fad0 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.993+0000 7fbdebe00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa20fad0 0x5621aa211ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.993+0000 7fbdebe00700 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa20fad0 0x5621aa211ec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccb001b03bc0ef52 server_cookie=1af61574381d2e7c in_seq=0 out_seq=0 2024-06-02T15:54:37.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.993+0000 7fbdd2a00700 10 client.26600 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.994+0000 7fbdd2a00700 1 -- 172.21.15.90:0/2058361379 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5621aa156d90 con 0x5621aa20fad0 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.994+0000 7fbdd2a00700 10 client.26600 handle_command_reply: tid=0 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.994+0000 7fbdfe266b80 1 client.26600 shutdown 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 msgr2=0x5621aa2024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009350 tx=0x7fbde000b410 comp rx=0 tx=0).stop 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 shutdown_connections 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa20fad0 0x5621aa211ec0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa1ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa2024b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdd2a00700 0 client.26600 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/2058361379 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa1ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 >> 172.21.15.90:0/2058361379 conn(0x5621aa124be0 msgr2=0x5621aa1f5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:37.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 shutdown_connections 2024-06-02T15:54:37.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 1 -- 172.21.15.90:0/2058361379 wait complete. 2024-06-02T15:54:37.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:37.995+0000 7fbdfe266b80 20 client.26600 trim_cache size 0 max 16384 2024-06-02T15:54:38.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.015+0000 7fbdfe266b80 1 Processor -- start 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.015+0000 7fbdfe266b80 1 -- start start 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 0x5621aa1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 0x5621aa1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b8360 con 0x5621aa1fc7d0 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b5b30 con 0x5621aa1fbeb0 2024-06-02T15:54:38.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b3980 con 0x5621aa1f6880 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 0x5621aa1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 0x5621aa1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdebe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 0x5621aa1f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35232/0 (socket says 172.21.15.90:35232) 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdebe00700 1 -- 172.21.15.90:0/1420451682 learned_addr learned my addr 172.21.15.90:0/1420451682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.016+0000 7fbdf1800700 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 -- 172.21.15.90:0/1420451682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 msgr2=0x5621aa1f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:38.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 0x5621aa1f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 -- 172.21.15.90:0/1420451682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 msgr2=0x5621aa1f9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 -- 172.21.15.90:0/1420451682 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621aa202e30 con 0x5621aa1f6880 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf1800700 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 0x5621aa1f6260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbde002ec70 tx=0x7fbde000baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27bd55125f437943 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbddd600700 1 -- 172.21.15.90:0/1420451682 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde002ee60 con 0x5621aa1f6880 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbddd600700 1 -- 172.21.15.90:0/1420451682 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0028040 con 0x5621aa1f6880 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.017+0000 7fbddd600700 1 -- 172.21.15.90:0/1420451682 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0033040 con 0x5621aa1f6880 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 -- 172.21.15.90:0/1420451682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 msgr2=0x5621aa1f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 0x5621aa1f6260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbde002ec70 tx=0x7fbde000baa0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 -- 172.21.15.90:0/1420451682 shutdown_connections 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fc7d0 0x5621aa1f9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1f6880 0x5621aa1f6260 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/1420451682 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1fbeb0 0x5621aa1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 -- 172.21.15.90:0/1420451682 >> 172.21.15.90:0/1420451682 conn(0x5621aa124be0 msgr2=0x5621aa1ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:38.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 -- 172.21.15.90:0/1420451682 shutdown_connections 2024-06-02T15:54:38.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.018+0000 7fbdfe266b80 1 -- 172.21.15.90:0/1420451682 wait complete. 2024-06-02T15:54:38.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.019+0000 7fbdfe266b80 1 -- start start 2024-06-02T15:54:38.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.019+0000 7fbdfe266b80 1 Processor -- start 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa212780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa212cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b3ff0 con 0x5621aa1fbeb0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b5460 con 0x5621aa1f6880 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1b46c0 con 0x5621aa1fc7d0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdfe266b80 10 client.0 _ll_register_callbacks cb 0x5621aa0eec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdf1800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa212cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdebe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa212780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdf0e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:35244/0 (socket says 172.21.15.90:35244) 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.020+0000 7fbdf0e00700 1 -- 172.21.15.90:0/658106996 learned_addr learned my addr 172.21.15.90:0/658106996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:38.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 -- 172.21.15.90:0/658106996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 msgr2=0x5621aa212cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa212cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 -- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 msgr2=0x5621aa212780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa212780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 -- 172.21.15.90:0/658106996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621aa2132b0 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdf0e00700 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005d90 tx=0x7fbde000bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1823039ce531692 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0033070 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde0028040 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.021+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fbde002e950 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbde004a050 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 10 client.26606 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5621aa27a110 con 0x5621aa1f6880 2024-06-02T15:54:38.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0028040 con 0x5621aa1f6880 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fbde0021070 con 0x5621aa1f6880 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 10 client.26606 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 10 client.26606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 20 client.26606 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa27a5e0 0x5621aa27ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 4 client.26606 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.022+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621aa150570 con 0x5621aa27a5e0 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.023+0000 7fbdebe00700 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa27a5e0 0x5621aa27ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.023+0000 7fbdebe00700 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa27a5e0 0x5621aa27ca00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da3ad035ad8f41f3 server_cookie=bc050c7a6edb18f6 in_seq=0 out_seq=0 2024-06-02T15:54:38.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.023+0000 7fbdd2a00700 10 client.26606 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdd2a00700 1 -- 172.21.15.90:0/658106996 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621aa150570 con 0x5621aa27a5e0 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdd2a00700 10 client.26606 handle_command_reply: tid=0 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 client.26606 shutdown 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 msgr2=0x5621aa215390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005d90 tx=0x7fbde000bde0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 shutdown_connections 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5621aa27a5e0 0x5621aa27ca00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621aa1fc7d0 0x5621aa212cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5621aa1f6880 0x5621aa215390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 --2- 172.21.15.90:0/658106996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621aa1fbeb0 0x5621aa212780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 >> 172.21.15.90:0/658106996 conn(0x5621aa124be0 msgr2=0x5621aa1f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdd2a00700 0 client.26606 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:54:38.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 shutdown_connections 2024-06-02T15:54:38.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.024+0000 7fbdfe266b80 1 -- 172.21.15.90:0/658106996 wait complete. 2024-06-02T15:54:38.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.025+0000 7fbdfe266b80 20 client.26606 trim_cache size 0 max 16384 2024-06-02T15:54:38.024 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:38.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.025+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 msgr2=0x7fbdd40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:38.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.025+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 0x7fbdd40b4d40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbde4009da0 tx=0x7fbde400b040 comp rx=0 tx=0).stop 2024-06-02T15:54:38.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.025+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 msgr2=0x7fbdec1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:38.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.025+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e9a0 tx=0x7fbdd800ecb0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 shutdown_connections 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fbdd40b2880 0x7fbdd40b4d40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdec114710 0x7fbdec1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec075770 0x7fbdec1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 --2- 172.21.15.90:0/1986359532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbdec076090 0x7fbdec1a0770 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 >> 172.21.15.90:0/1986359532 conn(0x7fbdec07a6a0 msgr2=0x7fbdec1115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 shutdown_connections 2024-06-02T15:54:38.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:38.026+0000 7fbdf3800700 1 -- 172.21.15.90:0/1986359532 wait complete. 2024-06-02T15:54:38.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:37 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:54:38.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:37 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:54:38.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:37 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:54:39.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:38 smithi090 ceph-mon[31595]: pgmap v824: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-02T15:54:39.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:38 smithi120 ceph-mon[36451]: pgmap v824: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-02T15:54:39.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:38 smithi135 ceph-mon[36839]: pgmap v824: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-06-02T15:54:39.381 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:54:39.826 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:40.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:54:40.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:54:40.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:39 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:54:40.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:54:40.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:54:40.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:39 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:54:40.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:54:40.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:54:40.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:39 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:54:41.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:40 smithi120 ceph-mon[36451]: pgmap v825: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:54:41.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:40 smithi135 ceph-mon[36839]: pgmap v825: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:54:41.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:40 smithi090 ceph-mon[31595]: pgmap v825: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 -- 172.21.15.90:0/3821621633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 msgr2=0x7fae58110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 --2- 172.21.15.90:0/3821621633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae58110380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae5000b0d0 tx=0x7fae5000b3e0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 -- 172.21.15.90:0/3821621633 shutdown_connections 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 --2- 172.21.15.90:0/3821621633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae58110380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 --2- 172.21.15.90:0/3821621633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae581095b0 0x7fae58109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 --2- 172.21.15.90:0/3821621633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae58103500 0x7fae58103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 -- 172.21.15.90:0/3821621633 >> 172.21.15.90:0/3821621633 conn(0x7fae580feff0 msgr2=0x7fae58101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 -- 172.21.15.90:0/3821621633 shutdown_connections 2024-06-02T15:54:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.366+0000 7fae5da00700 1 -- 172.21.15.90:0/3821621633 wait complete. 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.367+0000 7fae5da00700 1 Processor -- start 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 -- start start 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae581095b0 0x7fae5819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae58113700 con 0x7fae58103ec0 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae58113580 con 0x7fae58103500 2024-06-02T15:54:43.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae5da00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae58113880 con 0x7fae581095b0 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae4f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae57e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae57e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50596/0 (socket says 172.21.15.90:50596) 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae4f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47346/0 (socket says 172.21.15.90:47346) 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.368+0000 7fae57e00700 1 -- 172.21.15.90:0/2717922853 learned_addr learned my addr 172.21.15.90:0/2717922853 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae5d000700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae581095b0 0x7fae5819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 -- 172.21.15.90:0/2717922853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae581095b0 msgr2=0x7fae5819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae581095b0 0x7fae5819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 -- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 msgr2=0x7fae581a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 -- 172.21.15.90:0/2717922853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5819c450 con 0x7fae58103500 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae4f400700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.369+0000 7fae57e00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae4800eb30 tx=0x7fae4800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.370+0000 7fae55600700 1 -- 172.21.15.90:0/2717922853 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4800cbd0 con 0x7fae58103500 2024-06-02T15:54:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.370+0000 7fae55600700 1 -- 172.21.15.90:0/2717922853 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae480044d0 con 0x7fae58103500 2024-06-02T15:54:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.370+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5819c740 con 0x7fae58103500 2024-06-02T15:54:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.370+0000 7fae55600700 1 -- 172.21.15.90:0/2717922853 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48003e30 con 0x7fae58103500 2024-06-02T15:54:43.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.370+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a89d0 con 0x7fae58103500 2024-06-02T15:54:43.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.375+0000 7fae55600700 1 -- 172.21.15.90:0/2717922853 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae48004000 con 0x7fae58103500 2024-06-02T15:54:43.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 Processor -- start 2024-06-02T15:54:43.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 -- start start 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf56ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dcf5657e40 con 0x55dcf56fa390 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dcf56479e0 con 0x55dcf57000e0 2024-06-02T15:54:43.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.376+0000 7fae69a76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dcf564ba20 con 0x55dcf56ff7c0 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae4f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae57e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf56ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae4f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47350/0 (socket says 172.21.15.90:47350) 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae4f400700 1 -- 172.21.15.90:0/1990114538 learned_addr learned my addr 172.21.15.90:0/1990114538 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 -- 172.21.15.90:0/1990114538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 msgr2=0x55dcf56f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 -- 172.21.15.90:0/1990114538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 msgr2=0x55dcf56fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 -- 172.21.15.90:0/1990114538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf5610420 con 0x55dcf56ff7c0 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae55600700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 0x7fae3c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.377+0000 7fae5d000700 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf56ffba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae5000b770 tx=0x7fae50009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e9d372c4cb0269b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.378+0000 7fae57e00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 0x7fae3c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.378+0000 7fae55600700 1 -- 172.21.15.90:0/2717922853 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae480e28b0 con 0x7fae58103500 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.378+0000 7fae4d600700 1 -- 172.21.15.90:0/1990114538 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5000baa0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.378+0000 7fae4d600700 1 -- 172.21.15.90:0/1990114538 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae50020070 con 0x55dcf56ff7c0 2024-06-02T15:54:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.378+0000 7fae4d600700 1 -- 172.21.15.90:0/1990114538 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50005430 con 0x55dcf56ff7c0 2024-06-02T15:54:43.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 -- 172.21.15.90:0/1990114538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 msgr2=0x55dcf56ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae57e00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 0x7fae3c0b4dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c5b0 tx=0x7fae480e8550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf56ffba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae5000b770 tx=0x7fae50009b80 comp rx=0 tx=0).stop 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 -- 172.21.15.90:0/1990114538 shutdown_connections 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf56ffba0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 --2- 172.21.15.90:0/1990114538 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 -- 172.21.15.90:0/1990114538 >> 172.21.15.90:0/1990114538 conn(0x55dcf5627be0 msgr2=0x55dcf56f4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 -- 172.21.15.90:0/1990114538 shutdown_connections 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.379+0000 7fae69a76b80 1 -- 172.21.15.90:0/1990114538 wait complete. 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 -- start start 2024-06-02T15:54:43.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 Processor -- start 2024-06-02T15:54:43.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf56fa390 0x55dcf5705de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf57000e0 0x55dcf5703710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dcf554dfa0 con 0x55dcf57000e0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55dcf55e8760 con 0x55dcf56fa390 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dcf5706ba0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.380+0000 7fae69a76b80 10 client.0 _ll_register_callbacks cb 0x55dcf55f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:40570/0 (socket says 172.21.15.90:40570) 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 -- 172.21.15.90:0/2532525432 learned_addr learned my addr 172.21.15.90:0/2532525432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae5d000700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf57000e0 0x55dcf5703710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae5d000700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf56fa390 0x55dcf5705de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 -- 172.21.15.90:0/2532525432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf56fa390 msgr2=0x55dcf5705de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf56fa390 0x55dcf5705de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 -- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf57000e0 msgr2=0x55dcf5703710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf57000e0 0x55dcf5703710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 -- 172.21.15.90:0/2532525432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf55978d0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae4f400700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae40011d10 tx=0x7fae4000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299a654e8879222a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.381+0000 7fae46000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000fd90 con 0x55dcf56ff7c0 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae4001a3f0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae50020070 con 0x55dcf56ff7c0 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae40027050 con 0x55dcf56ff7c0 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae69a76b80 10 client.26551 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.382+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcf5618060 con 0x55dcf56ff7c0 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae50020070 con 0x55dcf56ff7c0 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fae40022030 con 0x55dcf56ff7c0 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 10 client.26551 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:43.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 10 client.26551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 20 client.26551 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf57137f0 0x55dcf5715be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 4 client.26551 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf5659d90 con 0x55dcf57137f0 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.383+0000 7fae5d000700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf57137f0 0x55dcf5715be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.384+0000 7fae46000700 10 client.26551 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:43.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.384+0000 7fae5d000700 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf57137f0 0x55dcf5715be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e963baa17f69459 server_cookie=dbe22840c9fc57d3 in_seq=0 out_seq=0 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae46000700 1 -- 172.21.15.90:0/2532525432 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dcf5659d90 con 0x55dcf57137f0 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae46000700 10 client.26551 handle_command_reply: tid=0 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 client.26551 shutdown 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 msgr2=0x55dcf57031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae40011d10 tx=0x7fae4000f6a0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 shutdown_connections 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf57000e0 0x55dcf5703710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf57137f0 0x55dcf5715be0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf56fa390 0x55dcf5705de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2532525432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf57031d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 >> 172.21.15.90:0/2532525432 conn(0x55dcf5627be0 msgr2=0x55dcf56f3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae46000700 0 client.26551 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.385+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 shutdown_connections 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.386+0000 7fae69a76b80 1 -- 172.21.15.90:0/2532525432 wait complete. 2024-06-02T15:54:43.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.386+0000 7fae69a76b80 20 client.26551 trim_cache size 0 max 16384 2024-06-02T15:54:43.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.402+0000 7fae69a76b80 1 Processor -- start 2024-06-02T15:54:43.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.402+0000 7fae69a76b80 1 -- start start 2024-06-02T15:54:43.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf5700bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b3c80 con 0x55dcf56fa390 2024-06-02T15:54:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b1450 con 0x55dcf57000e0 2024-06-02T15:54:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae69a76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae581af2a0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae5d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae5d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47370/0 (socket says 172.21.15.90:47370) 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae4f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf5700bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.403+0000 7fae5d000700 1 -- 172.21.15.90:0/3226089669 learned_addr learned my addr 172.21.15.90:0/3226089669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 -- 172.21.15.90:0/3226089669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 msgr2=0x55dcf5700bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf5700bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 -- 172.21.15.90:0/3226089669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 msgr2=0x55dcf56f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 -- 172.21.15.90:0/3226089669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf5705a20 con 0x55dcf56fa390 2024-06-02T15:54:43.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae5d000700 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae500062a0 tx=0x7fae500099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6c28c84623c7d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae4cc00700 1 -- 172.21.15.90:0/3226089669 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50005490 con 0x55dcf56fa390 2024-06-02T15:54:43.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae4cc00700 1 -- 172.21.15.90:0/3226089669 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae50020070 con 0x55dcf56fa390 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.404+0000 7fae4cc00700 1 -- 172.21.15.90:0/3226089669 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500077d0 con 0x55dcf56fa390 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 -- 172.21.15.90:0/3226089669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 msgr2=0x55dcf56f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae500062a0 tx=0x7fae500099e0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 -- 172.21.15.90:0/3226089669 shutdown_connections 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf56f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf56f9d70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 --2- 172.21.15.90:0/3226089669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf5700bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 -- 172.21.15.90:0/3226089669 >> 172.21.15.90:0/3226089669 conn(0x55dcf5627be0 msgr2=0x55dcf57004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 -- 172.21.15.90:0/3226089669 shutdown_connections 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.405+0000 7fae69a76b80 1 -- 172.21.15.90:0/3226089669 wait complete. 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 -- start start 2024-06-02T15:54:43.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 Processor -- start 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf571b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf571b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae581af910 con 0x55dcf56fa390 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b0d80 con 0x55dcf57000e0 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.406+0000 7fae69a76b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae581affe0 con 0x55dcf56ff7c0 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae69a76b80 10 client.0 _ll_register_callbacks cb 0x55dcf55f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47372/0 (socket says 172.21.15.90:47372) 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 -- 172.21.15.90:0/2144837328 learned_addr learned my addr 172.21.15.90:0/2144837328 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae4f400700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf571b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf571b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 -- 172.21.15.90:0/2144837328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 msgr2=0x55dcf571b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf571b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 -- 172.21.15.90:0/2144837328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 msgr2=0x55dcf571b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf571b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.407+0000 7fae5d000700 1 -- 172.21.15.90:0/2144837328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf577d360 con 0x55dcf56fa390 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae5d000700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fae50009850 tx=0x7fae50003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ffdd740a491a38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae46000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50039040 con 0x55dcf56fa390 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae5000bb60 con 0x55dcf56fa390 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae50020070 con 0x55dcf56fa390 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae5000bd60 con 0x55dcf56fa390 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae69a76b80 10 client.16323 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.408+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcf577e5c0 con 0x55dcf56fa390 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae50020070 con 0x55dcf56fa390 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fae50011070 con 0x55dcf56fa390 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 10 client.16323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 20 client.16323 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf577e890 0x55dcf5780c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 4 client.16323 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcf5653570 con 0x55dcf577e890 2024-06-02T15:54:43.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.409+0000 7fae4f400700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf577e890 0x55dcf5780c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:43.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae4f400700 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf577e890 0x55dcf5780c80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7bb62d37fbf954a server_cookie=85f24fc9b6e181af in_seq=0 out_seq=0 2024-06-02T15:54:43.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae46000700 10 client.16323 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:43.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae46000700 1 -- 172.21.15.90:0/2144837328 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcf5653570 con 0x55dcf577e890 2024-06-02T15:54:43.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae46000700 10 client.16323 handle_command_reply: tid=0 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae69a76b80 1 client.16323 shutdown 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 msgr2=0x55dcf571af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fae50009850 tx=0x7fae50003e90 comp rx=0 tx=0).stop 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 shutdown_connections 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.410+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55dcf577e890 0x55dcf5780c80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55dcf57000e0 0x55dcf571b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dcf56fa390 0x55dcf571af10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 --2- 172.21.15.90:0/2144837328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dcf56ff7c0 0x55dcf571b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae46000700 0 client.16323 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 >> 172.21.15.90:0/2144837328 conn(0x55dcf5627be0 msgr2=0x55dcf5700d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 shutdown_connections 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 1 -- 172.21.15.90:0/2144837328 wait complete. 2024-06-02T15:54:43.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.411+0000 7fae69a76b80 20 client.16323 trim_cache size 0 max 16384 2024-06-02T15:54:43.411 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:43.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 msgr2=0x7fae3c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 0x7fae3c0b4dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c5b0 tx=0x7fae480e8550 comp rx=0 tx=0).stop 2024-06-02T15:54:43.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 msgr2=0x7fae581a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae4800eb30 tx=0x7fae4800ee40 comp rx=0 tx=0).stop 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 shutdown_connections 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae3c0b2900 0x7fae3c0b4dc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae581095b0 0x7fae5819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae58103500 0x7fae581a1a50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 --2- 172.21.15.90:0/2717922853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae58103ec0 0x7fae581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 >> 172.21.15.90:0/2717922853 conn(0x7fae580feff0 msgr2=0x7fae581004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 shutdown_connections 2024-06-02T15:54:43.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:43.412+0000 7fae5da00700 1 -- 172.21.15.90:0/2717922853 wait complete. 2024-06-02T15:54:43.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:43 smithi120 ceph-mon[36451]: pgmap v826: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T15:54:43.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:43 smithi135 ceph-mon[36839]: pgmap v826: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T15:54:43.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:43 smithi090 ceph-mon[31595]: pgmap v826: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T15:54:44.740 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:54:45.193 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:45 smithi120 ceph-mon[36451]: pgmap v827: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:54:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:45 smithi135 ceph-mon[36839]: pgmap v827: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:54:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:45 smithi090 ceph-mon[31595]: pgmap v827: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:54:47.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:47 smithi090 ceph-mon[31595]: pgmap v828: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:47.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:47 smithi120 ceph-mon[36451]: pgmap v828: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:47.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:47 smithi135 ceph-mon[36839]: pgmap v828: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 -- 172.21.15.90:0/763250959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641088f0 msgr2=0x7f1e64110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 --2- 172.21.15.90:0/763250959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641088f0 0x7f1e64110470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00b0d0 tx=0x7f1e5c00b3e0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 -- 172.21.15.90:0/763250959 shutdown_connections 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 --2- 172.21.15.90:0/763250959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641088f0 0x7f1e64110470 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 --2- 172.21.15.90:0/763250959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e64103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 --2- 172.21.15.90:0/763250959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641041a0 0x7f1e641081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.045+0000 7f1e68a00700 1 -- 172.21.15.90:0/763250959 >> 172.21.15.90:0/763250959 conn(0x7f1e6407a6a0 msgr2=0x7f1e6407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.046+0000 7f1e68a00700 1 -- 172.21.15.90:0/763250959 shutdown_connections 2024-06-02T15:54:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.046+0000 7f1e68a00700 1 -- 172.21.15.90:0/763250959 wait complete. 2024-06-02T15:54:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.047+0000 7f1e68a00700 1 Processor -- start 2024-06-02T15:54:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.047+0000 7f1e68a00700 1 -- start start 2024-06-02T15:54:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641041a0 0x7f1e641a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641088f0 0x7f1e6419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641136c0 con 0x7f1e641088f0 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64113540 con 0x7f1e641041a0 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e68a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e64113840 con 0x7f1e641037f0 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e59400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641041a0 0x7f1e641a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e61e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:40614/0 (socket says 172.21.15.90:40614) 2024-06-02T15:54:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e61e00700 1 -- 172.21.15.90:0/2126900443 learned_addr learned my addr 172.21.15.90:0/2126900443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:48.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.048+0000 7f1e62800700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641088f0 0x7f1e6419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 -- 172.21.15.90:0/2126900443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641041a0 msgr2=0x7f1e641a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641041a0 0x7f1e641a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 -- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641088f0 msgr2=0x7f1e6419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641088f0 0x7f1e6419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 -- 172.21.15.90:0/2126900443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6419c260 con 0x7f1e641037f0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e61e00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d980 tx=0x7f1e5400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e5b400700 1 -- 172.21.15.90:0/2126900443 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54016070 con 0x7f1e641037f0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e5b400700 1 -- 172.21.15.90:0/2126900443 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e54004830 con 0x7f1e641037f0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.049+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6419c550 con 0x7f1e641037f0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.050+0000 7f1e5b400700 1 -- 172.21.15.90:0/2126900443 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540054c0 con 0x7f1e641037f0 2024-06-02T15:54:48.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.050+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641a8810 con 0x7f1e641037f0 2024-06-02T15:54:48.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.053+0000 7f1e5b400700 1 -- 172.21.15.90:0/2126900443 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e54004220 con 0x7f1e641037f0 2024-06-02T15:54:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.054+0000 7f1e7342bb80 1 Processor -- start 2024-06-02T15:54:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.054+0000 7f1e7342bb80 1 -- start start 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d6561eb0 0x5610d6562290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e5b400700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 0x7f1e480b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d655c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 0x5610d655cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610d64bae40 con 0x5610d65627d0 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610d64aa9e0 con 0x5610d655c880 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d64aea20 con 0x5610d6561eb0 2024-06-02T15:54:48.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e5b400700 1 -- 172.21.15.90:0/2126900443 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e540e24f0 con 0x7f1e641037f0 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e59400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d655c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e62800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 0x5610d655cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e62800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 0x5610d655cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50658/0 (socket says 172.21.15.90:50658) 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e62800700 1 -- 172.21.15.90:0/3771481606 learned_addr learned my addr 172.21.15.90:0/3771481606 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e62800700 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d6561eb0 0x5610d6562290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.055+0000 7f1e61e00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 0x7f1e480b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 -- 172.21.15.90:0/3771481606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d6561eb0 msgr2=0x5610d6562290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d6561eb0 0x5610d6562290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 -- 172.21.15.90:0/3771481606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 msgr2=0x5610d655cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 0x5610d655cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 -- 172.21.15.90:0/3771481606 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d6473420 con 0x5610d65627d0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e59400700 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d655c260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009c80 tx=0x7f1e4c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5fae7487eeaea8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e61e00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 0x7f1e480b4d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d950 tx=0x7f1e540076b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e53400700 1 -- 172.21.15.90:0/3771481606 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c011cb0 con 0x5610d65627d0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e53400700 1 -- 172.21.15.90:0/3771481606 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c011e50 con 0x5610d65627d0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.056+0000 7f1e53400700 1 -- 172.21.15.90:0/3771481606 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c005280 con 0x5610d65627d0 2024-06-02T15:54:48.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3771481606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 msgr2=0x5610d655c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d655c260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009c80 tx=0x7f1e4c009f90 comp rx=0 tx=0).stop 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3771481606 shutdown_connections 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d655c880 0x5610d655cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d6561eb0 0x5610d6562290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3771481606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d655c260 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.057+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3771481606 >> 172.21.15.90:0/3771481606 conn(0x5610d648abe0 msgr2=0x5610d6557010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.058+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3771481606 shutdown_connections 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.058+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3771481606 wait complete. 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.058+0000 7f1e7342bb80 1 -- start start 2024-06-02T15:54:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.058+0000 7f1e7342bb80 1 Processor -- start 2024-06-02T15:54:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d6565840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d65627d0 0x5610d6565d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610d63b0fa0 con 0x5610d655c880 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5610d644b760 con 0x5610d6561eb0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d6569240 con 0x5610d65627d0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e7342bb80 10 client.0 _ll_register_callbacks cb 0x5610d6454c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e62800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e59400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d6565840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e62800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47404/0 (socket says 172.21.15.90:47404) 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.059+0000 7f1e62800700 1 -- 172.21.15.90:0/1691047589 learned_addr learned my addr 172.21.15.90:0/1691047589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d65627d0 0x5610d6565d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 -- 172.21.15.90:0/1691047589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d65627d0 msgr2=0x5610d6565d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d65627d0 0x5610d6565d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 -- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 msgr2=0x5610d6565840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d6565840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 -- 172.21.15.90:0/1691047589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d63fa8d0 con 0x5610d655c880 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e62800700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00bfc0 tx=0x7f1e5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de67a76dd4a7246b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e47e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c007dc0 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e5c020450 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.060+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4c011e50 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1e5c020650 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e7342bb80 10 client.16335 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5610d647b060 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c011e50 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f1e5c011070 con 0x5610d655c880 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e7342bb80 10 client.16335 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e7342bb80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.061+0000 7f1e7342bb80 20 client.16335 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.062+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d657e4e0 0x5610d6580910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.062+0000 7f1e7342bb80 4 client.16335 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:48.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.062+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d64bcd90 con 0x5610d657e4e0 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.062+0000 7f1e59400700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d657e4e0 0x5610d6580910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.063+0000 7f1e59400700 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d657e4e0 0x5610d6580910 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9a772739a32c21 server_cookie=8b2c634371a29650 in_seq=0 out_seq=0 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.063+0000 7f1e47e00700 10 client.16335 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e47e00700 1 -- 172.21.15.90:0/1691047589 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5610d64bcd90 con 0x5610d657e4e0 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e47e00700 10 client.16335 handle_command_reply: tid=0 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 client.16335 shutdown 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 msgr2=0x5610d65684a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00bfc0 tx=0x7f1e5c009c00 comp rx=0 tx=0).stop 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 shutdown_connections 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d657e4e0 0x5610d6580910 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d65627d0 0x5610d6565d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d655c880 0x5610d65684a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/1691047589 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d6565840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e47e00700 0 client.16335 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 >> 172.21.15.90:0/1691047589 conn(0x5610d648abe0 msgr2=0x5610d655b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 shutdown_connections 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 1 -- 172.21.15.90:0/1691047589 wait complete. 2024-06-02T15:54:48.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.064+0000 7f1e7342bb80 20 client.16335 trim_cache size 0 max 16384 2024-06-02T15:54:48.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 Processor -- start 2024-06-02T15:54:48.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 -- start start 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d655c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 0x5610d6567cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641af9b0 con 0x5610d6561eb0 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641ad180 con 0x5610d65627d0 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.082+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641ab000 con 0x5610d655c880 2024-06-02T15:54:48.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47414/0 (socket says 172.21.15.90:47414) 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 -- 172.21.15.90:0/95008704 learned_addr learned my addr 172.21.15.90:0/95008704 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e62800700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d655c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e62800700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 0x5610d6567cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 -- 172.21.15.90:0/95008704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 msgr2=0x5610d655c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d655c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 -- 172.21.15.90:0/95008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 msgr2=0x5610d6567cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 0x5610d6567cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 -- 172.21.15.90:0/95008704 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d6550680 con 0x5610d6561eb0 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e62800700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 0x5610d6567cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:54:48.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.083+0000 7f1e59400700 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c014040 tx=0x7f1e4c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63042d57c29edff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.084+0000 7f1e52a00700 1 -- 172.21.15.90:0/95008704 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c029020 con 0x5610d6561eb0 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.084+0000 7f1e52a00700 1 -- 172.21.15.90:0/95008704 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c0044f0 con 0x5610d6561eb0 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.084+0000 7f1e52a00700 1 -- 172.21.15.90:0/95008704 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c003b20 con 0x5610d6561eb0 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.084+0000 7f1e7342bb80 1 -- 172.21.15.90:0/95008704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 msgr2=0x5610d65808d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.084+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c014040 tx=0x7f1e4c0092a0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 -- 172.21.15.90:0/95008704 shutdown_connections 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d65627d0 0x5610d6567cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d655c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/95008704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d6561eb0 0x5610d65808d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 -- 172.21.15.90:0/95008704 >> 172.21.15.90:0/95008704 conn(0x5610d648abe0 msgr2=0x5610d655f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 -- 172.21.15.90:0/95008704 shutdown_connections 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 -- 172.21.15.90:0/95008704 wait complete. 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 -- start start 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.085+0000 7f1e7342bb80 1 Processor -- start 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d6585c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d6586710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641ab640 con 0x5610d65627d0 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641acab0 con 0x5610d6561eb0 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e641abd10 con 0x5610d655c880 2024-06-02T15:54:48.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e7342bb80 10 client.0 _ll_register_callbacks cb 0x5610d6454c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e62800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d6585c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e59400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e62800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d6586710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e59400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:50682/0 (socket says 172.21.15.90:50682) 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.086+0000 7f1e59400700 1 -- 172.21.15.90:0/3429111868 learned_addr learned my addr 172.21.15.90:0/3429111868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 -- 172.21.15.90:0/3429111868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 msgr2=0x5610d6585c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d6585c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 -- 172.21.15.90:0/3429111868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 msgr2=0x5610d6586710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d6586710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 -- 172.21.15.90:0/3429111868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d65e8140 con 0x5610d6561eb0 2024-06-02T15:54:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e59400700 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009230 tx=0x7f1e4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91267752a43015c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e47e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c032070 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4c0044f0 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.087+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e4c025560 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1e4c025760 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 10 client.26639 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5610d65e92d0 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4c0044f0 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f1e4c018070 con 0x5610d6561eb0 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 10 client.26639 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 20 client.26639 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d65e9850 0x5610d65ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 4 client.26639 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:54:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.088+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610d64b6570 con 0x5610d65e9850 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e62800700 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d65e9850 0x5610d65ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e62800700 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d65e9850 0x5610d65ebc70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681c1f7e7c40891a server_cookie=4c3ddb0830f7bd1d in_seq=0 out_seq=0 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e47e00700 10 client.26639 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e47e00700 1 -- 172.21.15.90:0/3429111868 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610d64b6570 con 0x5610d65e9850 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e47e00700 10 client.26639 handle_command_reply: tid=0 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e7342bb80 1 client.26639 shutdown 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 msgr2=0x5610d65861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.089+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009230 tx=0x7f1e4c004730 comp rx=0 tx=0).stop 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 shutdown_connections 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5610d65e9850 0x5610d65ebc70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610d65627d0 0x5610d6586710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d655c880 0x5610d6585c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e47e00700 0 client.26639 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 --2- 172.21.15.90:0/3429111868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5610d6561eb0 0x5610d65861d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 >> 172.21.15.90:0/3429111868 conn(0x5610d648abe0 msgr2=0x5610d6564070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 shutdown_connections 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 1 -- 172.21.15.90:0/3429111868 wait complete. 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.090+0000 7f1e7342bb80 20 client.26639 trim_cache size 0 max 16384 2024-06-02T15:54:48.091 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 msgr2=0x7f1e480b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 0x7f1e480b4d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d950 tx=0x7f1e540076b0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 msgr2=0x7f1e641a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400d980 tx=0x7f1e5400dc90 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 shutdown_connections 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e480b2880 0x7f1e480b4d40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e641088f0 0x7f1e6419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e641037f0 0x7f1e641a1940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 --2- 172.21.15.90:0/2126900443 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e641041a0 0x7f1e641a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 >> 172.21.15.90:0/2126900443 conn(0x7f1e6407a6a0 msgr2=0x7f1e64102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 shutdown_connections 2024-06-02T15:54:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:48.091+0000 7f1e68a00700 1 -- 172.21.15.90:0/2126900443 wait complete. 2024-06-02T15:54:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:49 smithi120 ceph-mon[36451]: pgmap v829: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:54:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:49 smithi135 ceph-mon[36839]: pgmap v829: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:54:49.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:49 smithi090 ceph-mon[31595]: pgmap v829: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:54:50.216 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:54:50.217 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:54:50.665 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:51.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:51 smithi090 ceph-mon[31595]: pgmap v830: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:54:51.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:51 smithi120 ceph-mon[36451]: pgmap v830: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:54:51.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:51 smithi135 ceph-mon[36839]: pgmap v830: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:54:52.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:52 smithi090 ceph-mon[31595]: pgmap v831: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:52.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:52 smithi120 ceph-mon[36451]: pgmap v831: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:52.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:52 smithi135 ceph-mon[36839]: pgmap v831: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:54:53.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.480+0000 7f1dc1400700 1 -- 172.21.15.90:0/4048380566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc105760 msgr2=0x7f1dbc105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.480+0000 7f1dc1400700 1 --2- 172.21.15.90:0/4048380566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc105760 0x7f1dbc105be0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 -- 172.21.15.90:0/4048380566 shutdown_connections 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 --2- 172.21.15.90:0/4048380566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc106120 0x7f1dbc112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 --2- 172.21.15.90:0/4048380566 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 --2- 172.21.15.90:0/4048380566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc105760 0x7f1dbc105be0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 -- 172.21.15.90:0/4048380566 >> 172.21.15.90:0/4048380566 conn(0x7f1dbc101250 msgr2=0x7f1dbc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 -- 172.21.15.90:0/4048380566 shutdown_connections 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.481+0000 7f1dc1400700 1 -- 172.21.15.90:0/4048380566 wait complete. 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.482+0000 7f1dc1400700 1 Processor -- start 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.482+0000 7f1dc1400700 1 -- start start 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc105760 0x7f1dbc19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc106120 0x7f1dbc19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc115940 con 0x7f1dbc106120 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1157c0 con 0x7f1dbc10b760 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dc1400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc115ac0 con 0x7f1dbc105760 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1db9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc106120 0x7f1dbc19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dba800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc105760 0x7f1dbc19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42400/0 (socket says 172.21.15.90:42400) 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 -- 172.21.15.90:0/2168668695 learned_addr learned my addr 172.21.15.90:0/2168668695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 -- 172.21.15.90:0/2168668695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc105760 msgr2=0x7f1dbc19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc105760 0x7f1dbc19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 -- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc106120 msgr2=0x7f1dbc19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.483+0000 7f1dbb200700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc106120 0x7f1dbc19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1dbb200700 1 -- 172.21.15.90:0/2168668695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc1a0e00 con 0x7f1dbc10b760 2024-06-02T15:54:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1dbb200700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1db400ef30 tx=0x7f1db400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1db3400700 1 -- 172.21.15.90:0/2168668695 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40092e0 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc1a1090 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1db3400700 1 -- 172.21.15.90:0/2168668695 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4004bb0 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1db3400700 1 -- 172.21.15.90:0/2168668695 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4004020 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.484+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc1a8970 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.488+0000 7f1db3400700 1 -- 172.21.15.90:0/2168668695 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1db401b030 con 0x7f1dbc10b760 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 Processor -- start 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 -- start start 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c86640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562385be3ec0 con 0x562385c861c0 2024-06-02T15:54:53.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562385bd3a60 con 0x562385c8b690 2024-06-02T15:54:53.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.489+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562385bd7aa0 con 0x562385c8bfb0 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1db9e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60452/0 (socket says 172.21.15.90:60452) 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1db9e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42404/0 (socket says 172.21.15.90:42404) 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 -- 172.21.15.90:0/2417005748 learned_addr learned my addr 172.21.15.90:0/2417005748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:53.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c86640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 -- 172.21.15.90:0/2417005748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 msgr2=0x562385c8ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 -- 172.21.15.90:0/2417005748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 msgr2=0x562385c86640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c86640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1dba800700 1 -- 172.21.15.90:0/2417005748 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562385b9c4a0 con 0x562385c8bfb0 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.490+0000 7f1db3400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 0x7f1da00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1db3400700 1 -- 172.21.15.90:0/2168668695 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1db40e2080 con 0x7f1dbc10b760 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1dba800700 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009d00 tx=0x7f1dac011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fa2a80aa136428 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1da7e00700 1 -- 172.21.15.90:0/2417005748 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac004230 con 0x562385c8bfb0 2024-06-02T15:54:53.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 0x7f1da00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1da7e00700 1 -- 172.21.15.90:0/2417005748 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0043d0 con 0x562385c8bfb0 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.491+0000 7f1da7e00700 1 -- 172.21.15.90:0/2417005748 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0197e0 con 0x562385c8bfb0 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2417005748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 msgr2=0x562385c85ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 0x7f1da00b4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098e0 tx=0x7f1da8005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009d00 tx=0x7f1dac011720 comp rx=0 tx=0).stop 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2417005748 shutdown_connections 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c86640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2417005748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c85ba0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2417005748 >> 172.21.15.90:0/2417005748 conn(0x562385bb3c60 msgr2=0x562385c80710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.492+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2417005748 shutdown_connections 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.493+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2417005748 wait complete. 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.493+0000 7f1dcbe2fb80 1 -- start start 2024-06-02T15:54:53.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.493+0000 7f1dcbe2fb80 1 Processor -- start 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8bfb0 0x562385c8f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562385ada020 con 0x562385c861c0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562385b747e0 con 0x562385c8bfb0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562385c92a60 con 0x562385c8b690 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dcbe2fb80 10 client.0 _ll_register_callbacks cb 0x562385b7dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1db9e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40236/0 (socket says 172.21.15.90:40236) 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.494+0000 7f1dba800700 1 -- 172.21.15.90:0/974048429 learned_addr learned my addr 172.21.15.90:0/974048429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dbb200700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8bfb0 0x562385c8f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 -- 172.21.15.90:0/974048429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 msgr2=0x562385c8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 -- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8bfb0 msgr2=0x562385c8f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8bfb0 0x562385c8f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 -- 172.21.15.90:0/974048429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562385b23950 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1dba800700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1dac000c00 tx=0x7f1dac0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b11f469c36d714 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.495+0000 7f1da4c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac005240 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1dac0053e0 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac0043d0 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1dac033050 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1dcbe2fb80 10 client.16353 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562385ba40e0 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.496+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac0043d0 con 0x562385c861c0 2024-06-02T15:54:53.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f1dac022430 con 0x562385c861c0 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 20 client.16353 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385c9f550 0x562385ca1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 4 client.16353 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562385be5e10 con 0x562385c9f550 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.497+0000 7f1db9e00700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385c9f550 0x562385ca1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.498+0000 7f1db9e00700 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385c9f550 0x562385ca1940 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04777e1070492a4 server_cookie=7d5b2f21e57bcea8 in_seq=0 out_seq=0 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.498+0000 7f1da4c00700 10 client.16353 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:53.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1da4c00700 1 -- 172.21.15.90:0/974048429 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562385be5e10 con 0x562385c9f550 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1da4c00700 10 client.16353 handle_command_reply: tid=0 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 client.16353 shutdown 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 msgr2=0x562385c91cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1dac000c00 tx=0x7f1dac0038b0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 shutdown_connections 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385c9f550 0x562385ca1940 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8bfb0 0x562385c8f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c91cf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/974048429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1da4c00700 0 client.16353 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 >> 172.21.15.90:0/974048429 conn(0x562385bb3c60 msgr2=0x562385c7fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 shutdown_connections 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.499+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/974048429 wait complete. 2024-06-02T15:54:53.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.500+0000 7f1dcbe2fb80 20 client.16353 trim_cache size 0 max 16384 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 Processor -- start 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 -- start start 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c8bfb0 0x562385c80570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b3f80 con 0x562385c8bfb0 2024-06-02T15:54:53.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b1750 con 0x562385c861c0 2024-06-02T15:54:53.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.523+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1af5a0 con 0x562385c8b690 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42416/0 (socket says 172.21.15.90:42416) 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 -- 172.21.15.90:0/3209369257 learned_addr learned my addr 172.21.15.90:0/3209369257 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1db9e00700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dbb200700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c8bfb0 0x562385c80570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 -- 172.21.15.90:0/3209369257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 msgr2=0x562385c8cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 -- 172.21.15.90:0/3209369257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c8bfb0 msgr2=0x562385c80570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c8bfb0 0x562385c80570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 -- 172.21.15.90:0/3209369257 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562385c8bcc0 con 0x562385c861c0 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1db9e00700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:53.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.524+0000 7f1dba800700 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0095f0 tx=0x7f1dac005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7c9cf4ca592ae8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.525+0000 7f1da7400700 1 -- 172.21.15.90:0/3209369257 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac015070 con 0x562385c861c0 2024-06-02T15:54:53.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.525+0000 7f1da7400700 1 -- 172.21.15.90:0/3209369257 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac003cb0 con 0x562385c861c0 2024-06-02T15:54:53.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.525+0000 7f1da7400700 1 -- 172.21.15.90:0/3209369257 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0198d0 con 0x562385c861c0 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/3209369257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 msgr2=0x562385c85ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0095f0 tx=0x7f1dac005a00 comp rx=0 tx=0).stop 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/3209369257 shutdown_connections 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c8bfb0 0x562385c80570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c861c0 0x562385c85ba0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/3209369257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8b690 0x562385c8cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/3209369257 >> 172.21.15.90:0/3209369257 conn(0x562385bb3c60 msgr2=0x562385c8c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/3209369257 shutdown_connections 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/3209369257 wait complete. 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.526+0000 7f1dcbe2fb80 1 -- start start 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 Processor -- start 2024-06-02T15:54:53.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c98200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c94c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1afc10 con 0x562385c861c0 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b1080 con 0x562385c8b690 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc1b02e0 con 0x562385c8bfb0 2024-06-02T15:54:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.527+0000 7f1dcbe2fb80 10 client.0 _ll_register_callbacks cb 0x562385b7dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40252/0 (socket says 172.21.15.90:40252) 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dba800700 1 -- 172.21.15.90:0/2457235759 learned_addr learned my addr 172.21.15.90:0/2457235759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c98200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dbb200700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c94c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 -- 172.21.15.90:0/2457235759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 msgr2=0x562385c94c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c94c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 -- 172.21.15.90:0/2457235759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 msgr2=0x562385c97cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 -- 172.21.15.90:0/2457235759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562385c952d0 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dba800700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1dbb200700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c94c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1db9e00700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c98200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1da80059d0 tx=0x7f1da8005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ea2d1720c48739 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.528+0000 7f1da4c00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da800bc70 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac003cb0 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1da800be10 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1da8046050 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1dcbe2fb80 10 client.26657 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562385d0a910 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.529+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac003cb0 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f1da800f760 con 0x562385c8b690 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 10 client.26657 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:53.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:53.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 20 client.26657 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:53.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385d0b060 0x562385d0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:53.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 4 client.26657 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:54:53.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562385bdf5f0 con 0x562385d0b060 2024-06-02T15:54:53.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.530+0000 7f1dba800700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385d0b060 0x562385d0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.531+0000 7f1dba800700 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385d0b060 0x562385d0d480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e554f079419f2b59 server_cookie=c7c740eab5d420b2 in_seq=0 out_seq=0 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.531+0000 7f1da4c00700 10 client.26657 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1da4c00700 1 -- 172.21.15.90:0/2457235759 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562385bdf5f0 con 0x562385d0b060 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1da4c00700 10 client.26657 handle_command_reply: tid=0 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 client.26657 shutdown 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 msgr2=0x562385c98200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c98200 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1da80059d0 tx=0x7f1da8005960 comp rx=0 tx=0).stop 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 shutdown_connections 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x562385d0b060 0x562385d0d480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562385c8bfb0 0x562385c94c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562385c861c0 0x562385c97cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 --2- 172.21.15.90:0/2457235759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562385c8b690 0x562385c98200 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1da4c00700 0 client.26657 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 >> 172.21.15.90:0/2457235759 conn(0x562385bb3c60 msgr2=0x562385c8cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 shutdown_connections 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.532+0000 7f1dcbe2fb80 1 -- 172.21.15.90:0/2457235759 wait complete. 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dcbe2fb80 20 client.26657 trim_cache size 0 max 16384 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "d601cf20-3d21-4638-89ee-d996f4801dd8", 2024-06-02T15:54:53.532 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 msgr2=0x7f1da00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 0x7f1da00b4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1da80098e0 tx=0x7f1da8005040 comp rx=0 tx=0).stop 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 msgr2=0x7f1dbc1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1db400ef30 tx=0x7f1db400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 shutdown_connections 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1da00b2880 0x7f1da00b4d40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dbc10b760 0x7f1dbc1a06c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dbc105760 0x7f1dbc19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 --2- 172.21.15.90:0/2168668695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dbc106120 0x7f1dbc19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:53.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.533+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 >> 172.21.15.90:0/2168668695 conn(0x7f1dbc101250 msgr2=0x7f1dbc101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:53.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.534+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 shutdown_connections 2024-06-02T15:54:53.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:53.534+0000 7f1dc1400700 1 -- 172.21.15.90:0/2168668695 wait complete. 2024-06-02T15:54:54.711 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:54:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:54:55.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:55 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:54:55.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:55 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:55 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:54:55.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:55 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:55 smithi090 ceph-mon[31595]: pgmap v832: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:54:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:55 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:54:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:55 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:55 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:54:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:55 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:55 smithi120 ceph-mon[36451]: pgmap v832: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:54:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:55 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:54:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:55 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:55 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:54:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:55 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:54:55.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:55 smithi135 ceph-mon[36839]: pgmap v832: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:54:57.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:57 smithi090 ceph-mon[31595]: pgmap v833: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:54:57.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:57 smithi120 ceph-mon[36451]: pgmap v833: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:54:57.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:57 smithi135 ceph-mon[36839]: pgmap v833: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:54:58.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 -- 172.21.15.90:0/3656319749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103590 msgr2=0x7ff754103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 --2- 172.21.15.90:0/3656319749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103590 0x7ff754103a10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff740009990 tx=0x7ff740009ca0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 -- 172.21.15.90:0/3656319749 shutdown_connections 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 --2- 172.21.15.90:0/3656319749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103f50 0x7ff754110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 --2- 172.21.15.90:0/3656319749 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 0x7ff754109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 --2- 172.21.15.90:0/3656319749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103590 0x7ff754103a10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 -- 172.21.15.90:0/3656319749 >> 172.21.15.90:0/3656319749 conn(0x7ff75407a6f0 msgr2=0x7ff75407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 -- 172.21.15.90:0/3656319749 shutdown_connections 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.170+0000 7ff758e00700 1 -- 172.21.15.90:0/3656319749 wait complete. 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.171+0000 7ff758e00700 1 Processor -- start 2024-06-02T15:54:58.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 -- start start 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103590 0x7ff75419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 0x7ff7541a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541137c0 con 0x7ff754103f50 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff754113640 con 0x7ff7541096a0 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff758e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff754113940 con 0x7ff754103590 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff751400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff751e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103590 0x7ff75419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff751400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40262/0 (socket says 172.21.15.90:40262) 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff751400700 1 -- 172.21.15.90:0/1507470747 learned_addr learned my addr 172.21.15.90:0/1507470747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.172+0000 7ff752800700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 0x7ff7541a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 -- 172.21.15.90:0/1507470747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103590 msgr2=0x7ff75419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103590 0x7ff75419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 -- 172.21.15.90:0/1507470747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 msgr2=0x7ff7541a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 0x7ff7541a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 -- 172.21.15.90:0/1507470747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7541a0ea0 con 0x7ff754103f50 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.173+0000 7ff751400700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff740000c00 tx=0x7ff7400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.174+0000 7ff74aa00700 1 -- 172.21.15.90:0/1507470747 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740027070 con 0x7ff754103f50 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.174+0000 7ff74aa00700 1 -- 172.21.15.90:0/1507470747 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff740022470 con 0x7ff754103f50 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.174+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7541a1130 con 0x7ff754103f50 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.174+0000 7ff74aa00700 1 -- 172.21.15.90:0/1507470747 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740017650 con 0x7ff754103f50 2024-06-02T15:54:58.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.174+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a8a10 con 0x7ff754103f50 2024-06-02T15:54:58.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.177+0000 7ff74aa00700 1 -- 172.21.15.90:0/1507470747 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff74002a070 con 0x7ff754103f50 2024-06-02T15:54:58.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.179+0000 7ff74aa00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 0x7ff7380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.179+0000 7ff74aa00700 1 -- 172.21.15.90:0/1507470747 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7ff7400e3350 con 0x7ff754103f50 2024-06-02T15:54:58.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.179+0000 7ff751e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 0x7ff7380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 Processor -- start 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 -- start start 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff751e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 0x7ff7380b4d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff744009be0 tx=0x7ff744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69658290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 0x563d69652260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 0x563d69652ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563d695b0e40 con 0x563d69657eb0 2024-06-02T15:54:58.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d695a09e0 con 0x563d69652880 2024-06-02T15:54:58.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.180+0000 7ff76381eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563d695a4a20 con 0x563d696587d0 2024-06-02T15:54:58.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69658290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 0x563d69652260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69658290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40272/0 (socket says 172.21.15.90:40272) 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751e00700 1 -- 172.21.15.90:0/1851497662 learned_addr learned my addr 172.21.15.90:0/1851497662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff752800700 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 0x563d69652ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 -- 172.21.15.90:0/1851497662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 msgr2=0x563d69652ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 0x563d69652ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 -- 172.21.15.90:0/1851497662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 msgr2=0x563d69658290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69658290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff751400700 1 -- 172.21.15.90:0/1851497662 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d69569420 con 0x563d696587d0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.181+0000 7ff752800700 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 0x563d69652ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.182+0000 7ff751400700 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 0x563d69652260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff7400aa2f0 tx=0x7ff740009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27143a35a5a2df59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.182+0000 7ff73f400700 1 -- 172.21.15.90:0/1851497662 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400aa5f0 con 0x563d696587d0 2024-06-02T15:54:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.182+0000 7ff73f400700 1 -- 172.21.15.90:0/1851497662 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7400ae9a0 con 0x563d696587d0 2024-06-02T15:54:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.182+0000 7ff73f400700 1 -- 172.21.15.90:0/1851497662 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400aa850 con 0x563d696587d0 2024-06-02T15:54:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 -- 172.21.15.90:0/1851497662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 msgr2=0x563d69652260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 0x563d69652260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff7400aa2f0 tx=0x7ff740009310 comp rx=0 tx=0).stop 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 -- 172.21.15.90:0/1851497662 shutdown_connections 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69652880 0x563d69652ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69658290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1851497662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d696587d0 0x563d69652260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 -- 172.21.15.90:0/1851497662 >> 172.21.15.90:0/1851497662 conn(0x563d69580be0 msgr2=0x563d6964d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 -- 172.21.15.90:0/1851497662 shutdown_connections 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.183+0000 7ff76381eb80 1 -- 172.21.15.90:0/1851497662 wait complete. 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 -- start start 2024-06-02T15:54:58.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 Processor -- start 2024-06-02T15:54:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d6965b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563d694a6fa0 con 0x563d69657eb0 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x563d69541760 con 0x563d696587d0 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563d6965f000 con 0x563d69652880 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.184+0000 7ff76381eb80 10 client.0 _ll_register_callbacks cb 0x563d6954ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60506/0 (socket says 172.21.15.90:60506) 2024-06-02T15:54:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751e00700 1 -- 172.21.15.90:0/190953 learned_addr learned my addr 172.21.15.90:0/190953 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d6965b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff752800700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 -- 172.21.15.90:0/190953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 msgr2=0x563d6965e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 -- 172.21.15.90:0/190953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 msgr2=0x563d6965beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751400700 1 -- 172.21.15.90:0/190953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d694f08d0 con 0x563d69657eb0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.185+0000 7ff751e00700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff751400700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d6965b970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff740006010 tx=0x7ff7400e3250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b865b0e756ee63c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff733e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400aa5f0 con 0x563d69657eb0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7ff7400aa850 con 0x563d69657eb0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7400ae9a0 con 0x563d69657eb0 2024-06-02T15:54:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff7400aaa70 con 0x563d69657eb0 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff76381eb80 10 client.16371 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563d69571060 con 0x563d69657eb0 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.186+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7400c4e50 con 0x563d69657eb0 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7ff740005110 con 0x563d69657eb0 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 10 client.16371 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 20 client.16371 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d6966b9f0 0x563d6966dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 4 client.16371 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:54:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d695b2d90 con 0x563d6966b9f0 2024-06-02T15:54:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.187+0000 7ff751e00700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d6966b9f0 0x563d6966dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.188+0000 7ff751e00700 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d6966b9f0 0x563d6966dde0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cd913fc6c4c6c server_cookie=49af925b24a6317c in_seq=0 out_seq=0 2024-06-02T15:54:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.188+0000 7ff733e00700 10 client.16371 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff733e00700 1 -- 172.21.15.90:0/190953 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563d695b2d90 con 0x563d6966b9f0 2024-06-02T15:54:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff733e00700 10 client.16371 handle_command_reply: tid=0 2024-06-02T15:54:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 client.16371 shutdown 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 msgr2=0x563d6965b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d6965b970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff740006010 tx=0x7ff7400e3250 comp rx=0 tx=0).stop 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 shutdown_connections 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d6966b9f0 0x563d6966dde0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d6965e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 --2- 172.21.15.90:0/190953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d6965b970 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff733e00700 0 client.16371 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 >> 172.21.15.90:0/190953 conn(0x563d69580be0 msgr2=0x563d69651250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 shutdown_connections 2024-06-02T15:54:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.189+0000 7ff76381eb80 1 -- 172.21.15.90:0/190953 wait complete. 2024-06-02T15:54:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.190+0000 7ff76381eb80 20 client.16371 trim_cache size 0 max 16384 2024-06-02T15:54:58.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 Processor -- start 2024-06-02T15:54:58.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 -- start start 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69668060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69652260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541b3e00 con 0x563d69657eb0 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541b15d0 con 0x563d696587d0 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.212+0000 7ff76381eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541af420 con 0x563d69652880 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff752800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69652260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff752800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:42470/0 (socket says 172.21.15.90:42470) 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff752800700 1 -- 172.21.15.90:0/1271049324 learned_addr learned my addr 172.21.15.90:0/1271049324 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69668060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 -- 172.21.15.90:0/1271049324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 msgr2=0x563d6965a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 -- 172.21.15.90:0/1271049324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 msgr2=0x563d69652260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69652260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.213+0000 7ff751e00700 1 -- 172.21.15.90:0/1271049324 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6966e3a0 con 0x563d69652880 2024-06-02T15:54:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.214+0000 7ff751e00700 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69668060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563d695a0a80 tx=0x7ff744014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531324d1cf617684 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.214+0000 7ff73ea00700 1 -- 172.21.15.90:0/1271049324 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74401f860 con 0x563d69652880 2024-06-02T15:54:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.214+0000 7ff73ea00700 1 -- 172.21.15.90:0/1271049324 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff744031070 con 0x563d69652880 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.214+0000 7ff73ea00700 1 -- 172.21.15.90:0/1271049324 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74402a410 con 0x563d69652880 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 -- 172.21.15.90:0/1271049324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 msgr2=0x563d69668060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69668060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563d695a0a80 tx=0x7ff744014000 comp rx=0 tx=0).stop 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 -- 172.21.15.90:0/1271049324 shutdown_connections 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d696587d0 0x563d6965a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69668060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1271049324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d69657eb0 0x563d69652260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 -- 172.21.15.90:0/1271049324 >> 172.21.15.90:0/1271049324 conn(0x563d69580be0 msgr2=0x563d6964c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 -- 172.21.15.90:0/1271049324 shutdown_connections 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.215+0000 7ff76381eb80 1 -- 172.21.15.90:0/1271049324 wait complete. 2024-06-02T15:54:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.216+0000 7ff76381eb80 1 -- start start 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.216+0000 7ff76381eb80 1 Processor -- start 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69657eb0 0x563d69673880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d696587d0 0x563d69673dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541afa90 con 0x563d696587d0 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541b0f00 con 0x563d69657eb0 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541b0160 con 0x563d69652880 2024-06-02T15:54:58.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff76381eb80 10 client.0 _ll_register_callbacks cb 0x563d6954ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60510/0 (socket says 172.21.15.90:60510) 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 -- 172.21.15.90:0/1021318607 learned_addr learned my addr 172.21.15.90:0/1021318607 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff752800700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d696587d0 0x563d69673dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751400700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69657eb0 0x563d69673880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 -- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69657eb0 msgr2=0x563d69673880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69657eb0 0x563d69673880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 -- 172.21.15.90:0/1021318607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d696587d0 msgr2=0x563d69673dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d696587d0 0x563d69673dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.217+0000 7ff751e00700 1 -- 172.21.15.90:0/1021318607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d69657b00 con 0x563d69652880 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff751e00700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff744009a80 tx=0x7ff744003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d379f08c075a95e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff733e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:54:58.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74401f860 con 0x563d69652880 2024-06-02T15:54:58.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff744031070 con 0x563d69652880 2024-06-02T15:54:58.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7ff74401fa00 con 0x563d69652880 2024-06-02T15:54:58.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff74401fc00 con 0x563d69652880 2024-06-02T15:54:58.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff76381eb80 10 client.26611 fetch_fsmap learned FSMap version 41 2024-06-02T15:54:58.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.218+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563d696d6880 con 0x563d69652880 2024-06-02T15:54:58.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff744031070 con 0x563d69652880 2024-06-02T15:54:58.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7ff744004750 con 0x563d69652880 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 10 client.26611 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 20 client.26611 populate_metadata read hostname 'smithi090' 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d696d6fa0 0x563d696d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 4 client.26611 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.219+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d695ac570 con 0x563d696d6fa0 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff751400700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d696d6fa0 0x563d696d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff751400700 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d696d6fa0 0x563d696d93c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e5b21fb28ea713 server_cookie=6451f71bf126713e in_seq=0 out_seq=0 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff733e00700 10 client.26611 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff733e00700 1 -- 172.21.15.90:0/1021318607 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d695ac570 con 0x563d696d6fa0 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff733e00700 10 client.26611 handle_command_reply: tid=0 2024-06-02T15:54:58.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.220+0000 7ff76381eb80 1 client.26611 shutdown 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 msgr2=0x563d69673340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff744009a80 tx=0x7ff744003680 comp rx=0 tx=0).stop 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 shutdown_connections 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x563d696d6fa0 0x563d696d93c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d696587d0 0x563d69673dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563d69652880 0x563d69673340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff733e00700 0 client.26611 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 --2- 172.21.15.90:0/1021318607 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x563d69657eb0 0x563d69673880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 >> 172.21.15.90:0/1021318607 conn(0x563d69580be0 msgr2=0x563d69656af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 shutdown_connections 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 1 -- 172.21.15.90:0/1021318607 wait complete. 2024-06-02T15:54:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.221+0000 7ff76381eb80 20 client.26611 trim_cache size 0 max 16384 2024-06-02T15:54:58.222 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:54:58.222 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:54:58.222 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:54:58.222 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 msgr2=0x7ff7380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 0x7ff7380b4d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff744009be0 tx=0x7ff744009260 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 msgr2=0x7ff75419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff740000c00 tx=0x7ff7400057c0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 shutdown_connections 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff7380b2880 0x7ff7380b4d40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7541096a0 0x7ff7541a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff754103590 0x7ff75419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 --2- 172.21.15.90:0/1507470747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff754103f50 0x7ff75419c3d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 >> 172.21.15.90:0/1507470747 conn(0x7ff75407a6f0 msgr2=0x7ff754075a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 shutdown_connections 2024-06-02T15:54:58.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:54:58.222+0000 7ff758e00700 1 -- 172.21.15.90:0/1507470747 wait complete. 2024-06-02T15:54:58.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:58 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:54:58.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:58 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:54:58.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:58 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:54:59.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:54:59 smithi090 ceph-mon[31595]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T15:54:59.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:54:59.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:54:59.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:54:59 smithi120 ceph-mon[36451]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T15:54:59.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:54:59 smithi135 ceph-mon[36839]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T15:55:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:00.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:00 smithi090 ceph-mon[31595]: pgmap v835: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:00.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:00 smithi120 ceph-mon[36451]: pgmap v835: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:00.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:00 smithi135 ceph-mon[36839]: pgmap v835: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:02.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 -- 172.21.15.90:0/624003093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 msgr2=0x7fa8dc10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:02.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 --2- 172.21.15.90:0/624003093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc10fa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80098f0 tx=0x7fa8c8009c00 comp rx=0 tx=0).stop 2024-06-02T15:55:02.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 -- 172.21.15.90:0/624003093 shutdown_connections 2024-06-02T15:55:02.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 --2- 172.21.15.90:0/624003093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 --2- 172.21.15.90:0/624003093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 --2- 172.21.15.90:0/624003093 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc10fa00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.609+0000 7fa8e2800700 1 -- 172.21.15.90:0/624003093 >> 172.21.15.90:0/624003093 conn(0x7fa8dc07a6a0 msgr2=0x7fa8dc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:02.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.610+0000 7fa8e2800700 1 -- 172.21.15.90:0/624003093 shutdown_connections 2024-06-02T15:55:02.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.610+0000 7fa8e2800700 1 -- 172.21.15.90:0/624003093 wait complete. 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.611+0000 7fa8e2800700 1 Processor -- start 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.611+0000 7fa8e2800700 1 -- start start 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.611+0000 7fa8e2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.611+0000 7fa8e2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8e2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:02.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8e2800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc115950 con 0x7fa8dc075770 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8e2800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc1157d0 con 0x7fa8dc076090 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8e2800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc115ad0 con 0x7fa8dc10ff40 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8db400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60374/0 (socket says 172.21.15.90:60374) 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8db400700 1 -- 172.21.15.90:0/5906341 learned_addr learned my addr 172.21.15.90:0/5906341 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8daa00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 -- 172.21.15.90:0/5906341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 msgr2=0x7fa8dc19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 -- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 msgr2=0x7fa8dc19bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.612+0000 7fa8dbe00700 1 -- 172.21.15.90:0/5906341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc1a0ea0 con 0x7fa8dc10ff40 2024-06-02T15:55:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8daa00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8db400700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8dbe00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc1a0760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ef90 tx=0x7fa8d400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40092e0 con 0x7fa8dc10ff40 2024-06-02T15:55:02.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d4004bb0 con 0x7fa8dc10ff40 2024-06-02T15:55:02.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc1a1130 con 0x7fa8dc10ff40 2024-06-02T15:55:02.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4004020 con 0x7fa8dc10ff40 2024-06-02T15:55:02.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.613+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1a8a10 con 0x7fa8dc10ff40 2024-06-02T15:55:02.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.614+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc06a230 con 0x7fa8dc10ff40 2024-06-02T15:55:02.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.617+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa8d401b030 con 0x7fa8dc10ff40 2024-06-02T15:55:02.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.618+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 0x7fa8c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:02.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.618+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa8d40e22b0 con 0x7fa8dc10ff40 2024-06-02T15:55:02.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.618+0000 7fa8db400700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 0x7fa8c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:02.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.619+0000 7fa8db400700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 0x7fa8c00b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009750 tx=0x7fa8cc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:02.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.621+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8d4022020 con 0x7fa8dc10ff40 2024-06-02T15:55:02.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.916+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8dc02e560 con 0x7fa8dc10ff40 2024-06-02T15:55:02.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.917+0000 7fa8d3e00700 1 -- 172.21.15.90:0/5906341 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11889 (secure 0 0 0) 0x7fa8d4010040 con 0x7fa8dc10ff40 2024-06-02T15:55:02.915 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:55:02.915 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:54:27.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.920+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 msgr2=0x7fa8c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.920+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 0x7fa8c00b4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009750 tx=0x7fa8cc009310 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.920+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 msgr2=0x7fa8dc1a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.920+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc1a0760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400ef90 tx=0x7fa8d400c3a0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 shutdown_connections 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fa8c00b2880 0x7fa8c00b4d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8dc10ff40 0x7fa8dc1a0760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8dc075770 0x7fa8dc19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 --2- 172.21.15.90:0/5906341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8dc076090 0x7fa8dc19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 >> 172.21.15.90:0/5906341 conn(0x7fa8dc07a6a0 msgr2=0x7fa8dc1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:02.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 shutdown_connections 2024-06-02T15:55:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:02.921+0000 7fa8e2800700 1 -- 172.21.15.90:0/5906341 wait complete. 2024-06-02T15:55:02.920 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-06-02T15:55:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:03 smithi120 ceph-mon[36451]: pgmap v836: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:03.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:03 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/5906341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:03 smithi135 ceph-mon[36839]: pgmap v836: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:03.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:03 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/5906341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:03.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:03 smithi090 ceph-mon[31595]: pgmap v836: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:03 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/5906341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:03.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:55:04.406 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:05.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:05 smithi120 ceph-mon[36451]: pgmap v837: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:05.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:05 smithi135 ceph-mon[36839]: pgmap v837: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:05 smithi090 ceph-mon[31595]: pgmap v837: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:07.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 -- 172.21.15.90:0/1870398810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4574075770 msgr2=0x7f4574075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 --2- 172.21.15.90:0/1870398810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4574075770 0x7f4574075b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4568009990 tx=0x7f4568009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 -- 172.21.15.90:0/1870398810 shutdown_connections 2024-06-02T15:55:07.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 --2- 172.21.15.90:0/1870398810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f457410ff40 0x7f4574112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 --2- 172.21.15.90:0/1870398810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4574075770 0x7f4574075b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 --2- 172.21.15.90:0/1870398810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574076090 0x7f457410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.084+0000 7f457c800700 1 -- 172.21.15.90:0/1870398810 >> 172.21.15.90:0/1870398810 conn(0x7f457407a6a0 msgr2=0x7f457407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.085+0000 7f457c800700 1 -- 172.21.15.90:0/1870398810 shutdown_connections 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.085+0000 7f457c800700 1 -- 172.21.15.90:0/1870398810 wait complete. 2024-06-02T15:55:07.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 Processor -- start 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 -- start start 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 0x7f457419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 0x7f45741a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4574115910 con 0x7f457410ff40 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4574115790 con 0x7f4574076090 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f457c800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4574115a90 con 0x7f4574075770 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.086+0000 7f4579e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 0x7f457419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f457a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 0x7f45741a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60292/0 (socket says 172.21.15.90:60292) 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579e00700 1 -- 172.21.15.90:0/2473038300 learned_addr learned my addr 172.21.15.90:0/2473038300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 -- 172.21.15.90:0/2473038300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 msgr2=0x7f457419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 -- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 msgr2=0x7f45741a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 0x7f45741a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 -- 172.21.15.90:0/2473038300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45741a0e70 con 0x7f4574076090 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f457a800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 0x7f45741a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579e00700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.087+0000 7f4579400700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 0x7f457419c3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9c0 tx=0x7f456400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.088+0000 7f456ea00700 1 -- 172.21.15.90:0/2473038300 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564004230 con 0x7f4574076090 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.088+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45741a1160 con 0x7f4574076090 2024-06-02T15:55:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.088+0000 7f456ea00700 1 -- 172.21.15.90:0/2473038300 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640043d0 con 0x7f4574076090 2024-06-02T15:55:07.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.088+0000 7f456ea00700 1 -- 172.21.15.90:0/2473038300 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564003d90 con 0x7f4574076090 2024-06-02T15:55:07.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.088+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45741a89e0 con 0x7f4574076090 2024-06-02T15:55:07.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.092+0000 7f456ea00700 1 -- 172.21.15.90:0/2473038300 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f456400c7c0 con 0x7f4574076090 2024-06-02T15:55:07.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.092+0000 7f45870eab80 1 Processor -- start 2024-06-02T15:55:07.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 -- start start 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a50b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b779aee40 con 0x555b77a51140 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b7799e9e0 con 0x555b77a56610 2024-06-02T15:55:07.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.093+0000 7f45870eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555b779a2a20 con 0x555b77a56f30 2024-06-02T15:55:07.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f4579e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a50b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:40152/0 (socket says 172.21.15.90:40152) 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 -- 172.21.15.90:0/1273397532 learned_addr learned my addr 172.21.15.90:0/1273397532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f4579400700 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f456ea00700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 0x7f455c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 -- 172.21.15.90:0/1273397532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 msgr2=0x555b77a50b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a50b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 -- 172.21.15.90:0/1273397532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 msgr2=0x555b77a515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f457a800700 1 -- 172.21.15.90:0/1273397532 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b77967420 con 0x555b77a56610 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.094+0000 7f456ea00700 1 -- 172.21.15.90:0/2473038300 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f45640e28a0 con 0x7f4574076090 2024-06-02T15:55:07.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.095+0000 7f457a800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 0x7f455c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.095+0000 7f457a800700 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4570003e40 tx=0x7f4570009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d9812f7ffd6ae0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.095+0000 7f4563400700 1 -- 172.21.15.90:0/1273397532 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457001b040 con 0x555b77a56610 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.095+0000 7f4563400700 1 -- 172.21.15.90:0/1273397532 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457001c030 con 0x555b77a56610 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.095+0000 7f4563400700 1 -- 172.21.15.90:0/1273397532 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570015670 con 0x555b77a56610 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 -- 172.21.15.90:0/1273397532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 msgr2=0x555b77a569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f457a800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 0x7f455c0b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4570020000 tx=0x7f4570023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4570003e40 tx=0x7f4570009920 comp rx=0 tx=0).stop 2024-06-02T15:55:07.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 -- 172.21.15.90:0/1273397532 shutdown_connections 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a569f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 --2- 172.21.15.90:0/1273397532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a50b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.096+0000 7f45870eab80 1 -- 172.21.15.90:0/1273397532 >> 172.21.15.90:0/1273397532 conn(0x555b7797ebe0 msgr2=0x555b77a4b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.097+0000 7f45870eab80 1 -- 172.21.15.90:0/1273397532 shutdown_connections 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.097+0000 7f45870eab80 1 -- 172.21.15.90:0/1273397532 wait complete. 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.097+0000 7f45870eab80 1 -- start start 2024-06-02T15:55:07.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.097+0000 7f45870eab80 1 Processor -- start 2024-06-02T15:55:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a5cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a5a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a5a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b778a4fa0 con 0x555b77a51140 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555b7793f760 con 0x555b77a56610 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555b77a5d9e0 con 0x555b77a56f30 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f45870eab80 10 client.0 _ll_register_callbacks cb 0x555b77948c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f4579e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a5cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f4579400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a5a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f4579e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a5cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60400/0 (socket says 172.21.15.90:60400) 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f4579e00700 1 -- 172.21.15.90:0/3637010125 learned_addr learned my addr 172.21.15.90:0/3637010125 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.098+0000 7f457a800700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a5a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 -- 172.21.15.90:0/3637010125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 msgr2=0x555b77a5a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a5a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 -- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 msgr2=0x555b77a5cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a5cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 -- 172.21.15.90:0/3637010125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b778ee8d0 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4579400700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a5a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45640e0040 tx=0x7f45640a99d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67796662887db25b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4557e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640b22b0 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f45640a9d70 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.099+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457001c030 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f45640a9620 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 10 client.26693 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555b7796f060 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f457001c030 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f45640b24e0 con 0x555b77a56610 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 10 client.26693 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 10 client.26693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:55:07.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 20 client.26693 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:07.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77a72da0 0x555b77a751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 4 client.26693 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:07.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.100+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b779b0d90 con 0x555b77a72da0 2024-06-02T15:55:07.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.101+0000 7f4579e00700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77a72da0 0x555b77a751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.101+0000 7f4579e00700 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77a72da0 0x555b77a751b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca57a32a55d7082e server_cookie=712babb48f7ebc01 in_seq=0 out_seq=0 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.101+0000 7f4557e00700 10 client.26693 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.102+0000 7f4557e00700 1 -- 172.21.15.90:0/3637010125 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555b779b0d90 con 0x555b77a72da0 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.102+0000 7f4557e00700 10 client.26693 handle_command_reply: tid=0 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.102+0000 7f45870eab80 1 client.26693 shutdown 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 msgr2=0x555b77a5a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a5a010 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f45640e0040 tx=0x7f45640a99d0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 shutdown_connections 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77a72da0 0x555b77a751b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a5a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a5cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 --2- 172.21.15.90:0/3637010125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a5a010 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 >> 172.21.15.90:0/3637010125 conn(0x555b7797ebe0 msgr2=0x555b77a4ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f4557e00700 0 client.26693 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 shutdown_connections 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 1 -- 172.21.15.90:0/3637010125 wait complete. 2024-06-02T15:55:07.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.103+0000 7f45870eab80 20 client.26693 trim_cache size 0 max 16384 2024-06-02T15:55:07.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.120+0000 7f45870eab80 1 Processor -- start 2024-06-02T15:55:07.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.120+0000 7f45870eab80 1 -- start start 2024-06-02T15:55:07.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a51140 0x555b77a4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 0x555b77a7b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45741afb80 con 0x555b77a56f30 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45741ad350 con 0x555b77a51140 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f45870eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45741ab1d0 con 0x555b77a56610 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f457a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 0x555b77a7b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f457a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 0x555b77a7b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60408/0 (socket says 172.21.15.90:60408) 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60326/0 (socket says 172.21.15.90:60326) 2024-06-02T15:55:07.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f457a800700 1 -- 172.21.15.90:0/2705741804 learned_addr learned my addr 172.21.15.90:0/2705741804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579e00700 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a51140 0x555b77a4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 -- 172.21.15.90:0/2705741804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a51140 msgr2=0x555b77a4b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a51140 0x555b77a4b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 -- 172.21.15.90:0/2705741804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 msgr2=0x555b77a7b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 0x555b77a7b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.121+0000 7f4579400700 1 -- 172.21.15.90:0/2705741804 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b77a43620 con 0x555b77a56610 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.122+0000 7f4579400700 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45640a5b90 tx=0x7f45640e2c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba4dd459ae388b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.122+0000 7f4562a00700 1 -- 172.21.15.90:0/2705741804 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640b22b0 con 0x555b77a56610 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.122+0000 7f4562a00700 1 -- 172.21.15.90:0/2705741804 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640aa210 con 0x555b77a56610 2024-06-02T15:55:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.122+0000 7f4562a00700 1 -- 172.21.15.90:0/2705741804 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640adec0 con 0x555b77a56610 2024-06-02T15:55:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- 172.21.15.90:0/2705741804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 msgr2=0x555b77a7aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45640a5b90 tx=0x7f45640e2c00 comp rx=0 tx=0).stop 2024-06-02T15:55:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- 172.21.15.90:0/2705741804 shutdown_connections 2024-06-02T15:55:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a56f30 0x555b77a7b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a51140 0x555b77a4b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 --2- 172.21.15.90:0/2705741804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56610 0x555b77a7aac0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- 172.21.15.90:0/2705741804 >> 172.21.15.90:0/2705741804 conn(0x555b7797ebe0 msgr2=0x555b77a58940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- 172.21.15.90:0/2705741804 shutdown_connections 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- 172.21.15.90:0/2705741804 wait complete. 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.123+0000 7f45870eab80 1 -- start start 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 Processor -- start 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a6bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45741ab810 con 0x555b77a51140 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45741acc80 con 0x555b77a56610 2024-06-02T15:55:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45741abee0 con 0x555b77a56f30 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f45870eab80 10 client.0 _ll_register_callbacks cb 0x555b77948c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f457a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.124+0000 7f457a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60330/0 (socket says 172.21.15.90:60330) 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 -- 172.21.15.90:0/2167918973 learned_addr learned my addr 172.21.15.90:0/2167918973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f4579400700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a6bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 -- 172.21.15.90:0/2167918973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 msgr2=0x555b77a6bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a6bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 -- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 msgr2=0x555b77a6b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 -- 172.21.15.90:0/2167918973 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b77a7a690 con 0x555b77a56f30 2024-06-02T15:55:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f457a800700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4570025000 tx=0x7f45700049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=505f20a06932e8ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f4557e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.125+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570031030 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45640aa210 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f457000e4f0 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f457003e050 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 10 client.26635 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555b77add540 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640aa210 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f457002d950 con 0x555b77a56f30 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 10 client.26635 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 10 client.26635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:55:07.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.126+0000 7f45870eab80 20 client.26635 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:07.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77add970 0x555b77adfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:07.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f45870eab80 4 client.26635 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:07.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b779aa570 con 0x555b77add970 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f4579e00700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77add970 0x555b77adfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f4579e00700 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77add970 0x555b77adfd90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18d5845ef0753b96 server_cookie=e63f2c17e1d6839a in_seq=0 out_seq=0 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.127+0000 7f4557e00700 10 client.26635 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f4557e00700 1 -- 172.21.15.90:0/2167918973 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b779aa570 con 0x555b77add970 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f4557e00700 10 client.26635 handle_command_reply: tid=0 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 client.26635 shutdown 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 msgr2=0x555b77a79e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4570025000 tx=0x7f45700049e0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 shutdown_connections 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x555b77add970 0x555b77adfd90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555b77a56f30 0x555b77a79e50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b77a51140 0x555b77a6b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f4557e00700 0 client.26635 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 --2- 172.21.15.90:0/2167918973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555b77a56610 0x555b77a6bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 >> 172.21.15.90:0/2167918973 conn(0x555b7797ebe0 msgr2=0x555b77a573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 shutdown_connections 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.128+0000 7f45870eab80 1 -- 172.21.15.90:0/2167918973 wait complete. 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f45870eab80 20 client.26635 trim_cache size 0 max 16384 2024-06-02T15:55:07.133 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:07.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 msgr2=0x7f455c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 0x7f455c0b4d40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4570020000 tx=0x7f4570023000 comp rx=0 tx=0).stop 2024-06-02T15:55:07.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 msgr2=0x7f457419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:07.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 0x7f457419c3a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9c0 tx=0x7f456400ecd0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 shutdown_connections 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f455c0b2880 0x7f455c0b4d40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f457410ff40 0x7f45741a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4574075770 0x7f457419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 --2- 172.21.15.90:0/2473038300 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4574076090 0x7f457419c3a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.129+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 >> 172.21.15.90:0/2473038300 conn(0x7f457407a6a0 msgr2=0x7f4574101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.130+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 shutdown_connections 2024-06-02T15:55:07.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:07.130+0000 7f457c800700 1 -- 172.21.15.90:0/2473038300 wait complete. 2024-06-02T15:55:07.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:07 smithi120 ceph-mon[36451]: pgmap v838: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:55:07.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:07 smithi135 ceph-mon[36839]: pgmap v838: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:55:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:07 smithi090 ceph-mon[31595]: pgmap v838: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:55:08.120 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:55:08.575 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:09.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:09 smithi120 ceph-mon[36451]: pgmap v839: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-06-02T15:55:09.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:09 smithi135 ceph-mon[36839]: pgmap v839: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-06-02T15:55:09.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:09 smithi090 ceph-mon[31595]: pgmap v839: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-06-02T15:55:10.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:10 smithi120 ceph-mon[36451]: pgmap v840: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 8 op/s 2024-06-02T15:55:10.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:10 smithi135 ceph-mon[36839]: pgmap v840: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 8 op/s 2024-06-02T15:55:10.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:10 smithi090 ceph-mon[31595]: pgmap v840: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 8 op/s 2024-06-02T15:55:10.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 -- 172.21.15.90:0/1599875767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb441094f0 msgr2=0x7fcb441098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 --2- 172.21.15.90:0/1599875767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb441094f0 0x7fcb441098d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009f90 tx=0x7fcb3400b330 comp rx=0 tx=0).stop 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 -- 172.21.15.90:0/1599875767 shutdown_connections 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 --2- 172.21.15.90:0/1599875767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb44103e20 0x7fcb44110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 --2- 172.21.15.90:0/1599875767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb441094f0 0x7fcb441098d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 --2- 172.21.15.90:0/1599875767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103460 0x7fcb441038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 -- 172.21.15.90:0/1599875767 >> 172.21.15.90:0/1599875767 conn(0x7fcb440fef50 msgr2=0x7fcb44101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 -- 172.21.15.90:0/1599875767 shutdown_connections 2024-06-02T15:55:10.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.942+0000 7fcb43200700 1 -- 172.21.15.90:0/1599875767 wait complete. 2024-06-02T15:55:10.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.944+0000 7fcb43200700 1 Processor -- start 2024-06-02T15:55:10.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.944+0000 7fcb43200700 1 -- start start 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 0x7fcb441a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44113670 con 0x7fcb44103460 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441134f0 con 0x7fcb441094f0 2024-06-02T15:55:10.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb43200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441137f0 con 0x7fcb44103e20 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52104/0 (socket says 172.21.15.90:52104) 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44892/0 (socket says 172.21.15.90:44892) 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41e00700 1 -- 172.21.15.90:0/3951386432 learned_addr learned my addr 172.21.15.90:0/3951386432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb42800700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 0x7fcb441a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 -- 172.21.15.90:0/3951386432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 msgr2=0x7fcb441a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 0x7fcb441a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 -- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 msgr2=0x7fcb4419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.945+0000 7fcb41400700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb41400700 1 -- 172.21.15.90:0/3951386432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441a0d70 con 0x7fcb44103e20 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb42800700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 0x7fcb441a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:55:10.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb41e00700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb41400700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9f0 tx=0x7fcb3000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb3aa00700 1 -- 172.21.15.90:0/3951386432 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000cbd0 con 0x7fcb44103e20 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb441a1060 con 0x7fcb44103e20 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb3aa00700 1 -- 172.21.15.90:0/3951386432 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb300044d0 con 0x7fcb44103e20 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb3aa00700 1 -- 172.21.15.90:0/3951386432 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30005280 con 0x7fcb44103e20 2024-06-02T15:55:10.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.946+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb441a88e0 con 0x7fcb44103e20 2024-06-02T15:55:10.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.951+0000 7fcb53bf6b80 1 Processor -- start 2024-06-02T15:55:10.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.951+0000 7fcb53bf6b80 1 -- start start 2024-06-02T15:55:10.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb3aa00700 1 -- 172.21.15.90:0/3951386432 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcb30005420 con 0x7fcb44103e20 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11ed9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f11ecfce40 con 0x55f11ed9ead0 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f11ecec9e0 con 0x55f11eda4940 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f11ecf0a20 con 0x55f11eda4020 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11ed9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb42800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb42800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44904/0 (socket says 172.21.15.90:44904) 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb42800700 1 -- 172.21.15.90:0/3275371552 learned_addr learned my addr 172.21.15.90:0/3275371552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.952+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51070/0 (socket says 172.21.15.90:51070) 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 -- 172.21.15.90:0/3275371552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 msgr2=0x55f11eda4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 -- 172.21.15.90:0/3275371552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 msgr2=0x55f11ed9ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11ed9ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 -- 172.21.15.90:0/3275371552 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11ecb5420 con 0x55f11eda4940 2024-06-02T15:55:10.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb42800700 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:10.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb41e00700 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009790 tx=0x7fcb34004e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19476ec8dc573caf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.953+0000 7fcb2f400700 1 -- 172.21.15.90:0/3275371552 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400b9f0 con 0x55f11eda4940 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb2f400700 1 -- 172.21.15.90:0/3275371552 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3400bb90 con 0x55f11eda4940 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb2f400700 1 -- 172.21.15.90:0/3275371552 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340052a0 con 0x55f11eda4940 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb3aa00700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 0x7fcb280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb3aa00700 1 -- 172.21.15.90:0/3951386432 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcb300e2520 con 0x7fcb44103e20 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/3275371552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 msgr2=0x55f11ed9e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.954+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009790 tx=0x7fcb34004e80 comp rx=0 tx=0).stop 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb42800700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 0x7fcb280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/3275371552 shutdown_connections 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11ed9ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11eda4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/3275371552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11ed9e4b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/3275371552 >> 172.21.15.90:0/3275371552 conn(0x55f11ecccbe0 msgr2=0x55f11ed99150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/3275371552 shutdown_connections 2024-06-02T15:55:10.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/3275371552 wait complete. 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb42800700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 0x7fcb280b4d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009750 tx=0x7fcb3c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.955+0000 7fcb53bf6b80 1 -- start start 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 Processor -- start 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 0x55f11edaa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 0x55f11eda7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f11ebf2fa0 con 0x55f11eda4940 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f11ec8d760 con 0x55f11eda4020 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f11edab400 con 0x55f11ed9ead0 2024-06-02T15:55:10.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.956+0000 7fcb53bf6b80 10 client.0 _ll_register_callbacks cb 0x55f11ec96c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 0x55f11edaa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 0x55f11edaa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44908/0 (socket says 172.21.15.90:44908) 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41e00700 1 -- 172.21.15.90:0/2842783232 learned_addr learned my addr 172.21.15.90:0/2842783232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51084/0 (socket says 172.21.15.90:51084) 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb42800700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 0x55f11eda7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 -- 172.21.15.90:0/2842783232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 msgr2=0x55f11edaa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 0x55f11edaa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 -- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 msgr2=0x55f11eda7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 0x55f11eda7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb41400700 1 -- 172.21.15.90:0/2842783232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11ec3c8d0 con 0x55f11eda4020 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.957+0000 7fcb42800700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 0x55f11eda7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb41400700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb300dfcc0 tx=0x7fcb30004980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fae1ca1a131172 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb23e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb300a5650 con 0x55f11eda4020 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcb30091300 con 0x55f11eda4020 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3400bb90 con 0x55f11eda4020 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcb300a8f00 con 0x55f11eda4020 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb53bf6b80 10 client.26711 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f11ecbd060 con 0x55f11eda4020 2024-06-02T15:55:10.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.958+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3400bb90 con 0x55f11eda4020 2024-06-02T15:55:10.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fcb300bebe0 con 0x55f11eda4020 2024-06-02T15:55:10.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 10 client.26711 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:10.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 10 client.26711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 20 client.26711 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11edb8070 0x55f11edba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 4 client.26711 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11ecfed90 con 0x55f11edb8070 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.959+0000 7fcb41e00700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11edb8070 0x55f11edba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.960+0000 7fcb23e00700 10 client.26711 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.960+0000 7fcb41e00700 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11edb8070 0x55f11edba4d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc681f2ebe11e5a1 server_cookie=68c451eb3eaca2a3 in_seq=0 out_seq=0 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.960+0000 7fcb23e00700 1 -- 172.21.15.90:0/2842783232 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f11ecfed90 con 0x55f11edb8070 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.960+0000 7fcb23e00700 10 client.26711 handle_command_reply: tid=0 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 client.26711 shutdown 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 msgr2=0x55f11eda7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb300dfcc0 tx=0x7fcb30004980 comp rx=0 tx=0).stop 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 shutdown_connections 2024-06-02T15:55:10.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11eda4940 0x55f11eda7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11ed9ead0 0x55f11edaa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11edb8070 0x55f11edba4d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2842783232 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11eda7a60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb23e00700 0 client.26711 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 >> 172.21.15.90:0/2842783232 conn(0x55f11ecccbe0 msgr2=0x55f11ed9d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 shutdown_connections 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2842783232 wait complete. 2024-06-02T15:55:10.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.961+0000 7fcb53bf6b80 20 client.26711 trim_cache size 0 max 16384 2024-06-02T15:55:10.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.978+0000 7fcb53bf6b80 1 Processor -- start 2024-06-02T15:55:10.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.978+0000 7fcb53bf6b80 1 -- start start 2024-06-02T15:55:10.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11edb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441b3ca0 con 0x55f11ed9ead0 2024-06-02T15:55:10.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441b1470 con 0x55f11eda4020 2024-06-02T15:55:10.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441af2c0 con 0x55f11eda4940 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb42800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb42800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44920/0 (socket says 172.21.15.90:44920) 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb42800700 1 -- 172.21.15.90:0/2606697526 learned_addr learned my addr 172.21.15.90:0/2606697526 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.979+0000 7fcb41400700 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11edb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 -- 172.21.15.90:0/2606697526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 msgr2=0x55f11edb49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11edb49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 -- 172.21.15.90:0/2606697526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 msgr2=0x55f11edb44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edb44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 -- 172.21.15.90:0/2606697526 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11edbab50 con 0x55f11eda4940 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb41e00700 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb42800700 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c012f70 tx=0x7fcb3c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc6eff93c51e31f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb2ea00700 1 -- 172.21.15.90:0/2606697526 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c02d070 con 0x55f11eda4940 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb2ea00700 1 -- 172.21.15.90:0/2606697526 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c028410 con 0x55f11eda4940 2024-06-02T15:55:10.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.980+0000 7fcb2ea00700 1 -- 172.21.15.90:0/2606697526 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c01d400 con 0x55f11eda4940 2024-06-02T15:55:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2606697526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 msgr2=0x55f11eda1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c012f70 tx=0x7fcb3c017000 comp rx=0 tx=0).stop 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2606697526 shutdown_connections 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4940 0x55f11eda1b70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edb44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/2606697526 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4020 0x55f11edb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2606697526 >> 172.21.15.90:0/2606697526 conn(0x55f11ecccbe0 msgr2=0x55f11eda7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2606697526 shutdown_connections 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.981+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/2606697526 wait complete. 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.982+0000 7fcb53bf6b80 1 -- start start 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.982+0000 7fcb53bf6b80 1 Processor -- start 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11edbfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11edc0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441af930 con 0x55f11ed9ead0 2024-06-02T15:55:10.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441b0da0 con 0x55f11eda4940 2024-06-02T15:55:10.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441b0000 con 0x55f11eda4020 2024-06-02T15:55:10.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb53bf6b80 10 client.0 _ll_register_callbacks cb 0x55f11ec96c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:10.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52142/0 (socket says 172.21.15.90:52142) 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb41400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11edbfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb41e00700 1 -- 172.21.15.90:0/949558826 learned_addr learned my addr 172.21.15.90:0/949558826 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.983+0000 7fcb42800700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11edc0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 -- 172.21.15.90:0/949558826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 msgr2=0x55f11edc0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11edc0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 -- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 msgr2=0x55f11edbf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 -- 172.21.15.90:0/949558826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11ee21ce0 con 0x55f11eda4020 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb42800700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11edc0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41e00700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb41400700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11edbfe50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb300a5810 tx=0x7fcb300bee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a509531efa3d9a70 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb23e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb300a90e0 con 0x55f11eda4020 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c028410 con 0x55f11eda4020 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.984+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcb300a5d80 con 0x55f11eda4020 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcb300a9560 con 0x55f11eda4020 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 10 client.26653 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:10.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f11ee22fc0 con 0x55f11eda4020 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c028410 con 0x55f11eda4020 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fcb300a8460 con 0x55f11eda4020 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 10 client.26653 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 10 client.26653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 20 client.26653 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11ee235b0 0x55f11ee259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 4 client.26653 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.985+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f11ecf8570 con 0x55f11ee235b0 2024-06-02T15:55:10.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.986+0000 7fcb41e00700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11ee235b0 0x55f11ee259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:10.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.986+0000 7fcb41e00700 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11ee235b0 0x55f11ee259d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bff303c2d72eb173 server_cookie=4688ddf96d282592 in_seq=0 out_seq=0 2024-06-02T15:55:10.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.986+0000 7fcb23e00700 10 client.26653 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:10.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb23e00700 1 -- 172.21.15.90:0/949558826 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f11ecf8570 con 0x55f11ee235b0 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb23e00700 10 client.26653 handle_command_reply: tid=0 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 client.26653 shutdown 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 msgr2=0x55f11edbfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11edbfe50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb300a5810 tx=0x7fcb300bee10 comp rx=0 tx=0).stop 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 shutdown_connections 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55f11ee235b0 0x55f11ee259d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f11eda4940 0x55f11edc0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f11ed9ead0 0x55f11edbf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 --2- 172.21.15.90:0/949558826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f11eda4020 0x55f11edbfe50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb23e00700 0 client.26653 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 >> 172.21.15.90:0/949558826 conn(0x55f11ecccbe0 msgr2=0x55f11ed98c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 shutdown_connections 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 1 -- 172.21.15.90:0/949558826 wait complete. 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.987+0000 7fcb53bf6b80 20 client.26653 trim_cache size 0 max 16384 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:10.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 msgr2=0x7fcb280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 0x7fcb280b4d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009750 tx=0x7fcb3c009310 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 msgr2=0x7fcb4419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9f0 tx=0x7fcb3000ed00 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 shutdown_connections 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fcb280b2880 0x7fcb280b4d40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcb441094f0 0x7fcb441a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb44103460 0x7fcb4419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 --2- 172.21.15.90:0/3951386432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb44103e20 0x7fcb4419c2a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 >> 172.21.15.90:0/3951386432 conn(0x7fcb440fef50 msgr2=0x7fcb4410ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 shutdown_connections 2024-06-02T15:55:10.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:10.988+0000 7fcb43200700 1 -- 172.21.15.90:0/3951386432 wait complete. 2024-06-02T15:55:11.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:55:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:13.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:13 smithi120 ceph-mon[36451]: pgmap v841: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:13.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:13 smithi135 ceph-mon[36839]: pgmap v841: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:13.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:13 smithi090 ceph-mon[31595]: pgmap v841: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:15.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:15 smithi120 ceph-mon[36451]: pgmap v842: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:15.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:15 smithi135 ceph-mon[36839]: pgmap v842: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:15.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:15 smithi090 ceph-mon[31595]: pgmap v842: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 -- 172.21.15.90:0/3552188959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f969410fd10 msgr2=0x7f9694112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 --2- 172.21.15.90:0/3552188959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f969410fd10 0x7f9694112120 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9690009990 tx=0x7f9690009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 -- 172.21.15.90:0/3552188959 shutdown_connections 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 --2- 172.21.15.90:0/3552188959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f969410fd10 0x7f9694112120 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 --2- 172.21.15.90:0/3552188959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694075910 0x7f9694075cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 --2- 172.21.15.90:0/3552188959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9694076230 0x7f969410f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 -- 172.21.15.90:0/3552188959 >> 172.21.15.90:0/3552188959 conn(0x7f96940fefc0 msgr2=0x7f96941013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 -- 172.21.15.90:0/3552188959 shutdown_connections 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.295+0000 7f969b200700 1 -- 172.21.15.90:0/3552188959 wait complete. 2024-06-02T15:55:16.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 Processor -- start 2024-06-02T15:55:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 -- start start 2024-06-02T15:55:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969410fd10 0x7f96941a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96941158a0 con 0x7f9694075910 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9694115720 con 0x7f9694076230 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.297+0000 7f969b200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9694115a20 con 0x7f969410fd10 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f969a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969410fd10 0x7f96941a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52162/0 (socket says 172.21.15.90:52162) 2024-06-02T15:55:16.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699e00700 1 -- 172.21.15.90:0/3427424079 learned_addr learned my addr 172.21.15.90:0/3427424079 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51112/0 (socket says 172.21.15.90:51112) 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 -- 172.21.15.90:0/3427424079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969410fd10 msgr2=0x7f96941a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969410fd10 0x7f96941a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 -- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 msgr2=0x7f969419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699400700 1 -- 172.21.15.90:0/3427424079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96941a0d30 con 0x7f9694076230 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.298+0000 7f9699e00700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:16.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f9699400700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f968400e9f0 tx=0x7f968400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f968aa00700 1 -- 172.21.15.90:0/3427424079 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684004220 con 0x7f9694076230 2024-06-02T15:55:16.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96941a1020 con 0x7f9694076230 2024-06-02T15:55:16.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f968aa00700 1 -- 172.21.15.90:0/3427424079 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96840043c0 con 0x7f9694076230 2024-06-02T15:55:16.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f968aa00700 1 -- 172.21.15.90:0/3427424079 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684005020 con 0x7f9694076230 2024-06-02T15:55:16.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.299+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9694103360 con 0x7f9694076230 2024-06-02T15:55:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.303+0000 7f968aa00700 1 -- 172.21.15.90:0/3427424079 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9684004750 con 0x7f9694076230 2024-06-02T15:55:16.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.304+0000 7f96a71f0b80 1 Processor -- start 2024-06-02T15:55:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.304+0000 7f96a71f0b80 1 -- start start 2024-06-02T15:55:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad6116ad0 0x55fad6116f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fad6074e40 con 0x55fad6116ad0 2024-06-02T15:55:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fad60649e0 con 0x55fad611c020 2024-06-02T15:55:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fad6068a20 con 0x55fad611c940 2024-06-02T15:55:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f9699e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f969a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f9699400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad6116ad0 0x55fad6116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f9699e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44946/0 (socket says 172.21.15.90:44946) 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f969a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51128/0 (socket says 172.21.15.90:51128) 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.305+0000 7f9699e00700 1 -- 172.21.15.90:0/3306249662 learned_addr learned my addr 172.21.15.90:0/3306249662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f968aa00700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 0x7f96800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 -- 172.21.15.90:0/3306249662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 msgr2=0x55fad611c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 -- 172.21.15.90:0/3306249662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad6116ad0 msgr2=0x55fad6116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad6116ad0 0x55fad6116f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 -- 172.21.15.90:0/3306249662 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad602d420 con 0x55fad611c940 2024-06-02T15:55:16.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f969a800700 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:16.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f968aa00700 1 -- 172.21.15.90:0/3427424079 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f96840e24b0 con 0x7f9694076230 2024-06-02T15:55:16.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f9699e00700 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ec0 tx=0x7f968c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb707f98000d101 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f969a800700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 0x7f96800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f967f400700 1 -- 172.21.15.90:0/3306249662 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c019070 con 0x55fad611c940 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f967f400700 1 -- 172.21.15.90:0/3306249662 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c007dc0 con 0x55fad611c940 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.306+0000 7f967f400700 1 -- 172.21.15.90:0/3306249662 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c015d40 con 0x55fad611c940 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 -- 172.21.15.90:0/3306249662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 msgr2=0x55fad61164b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f969a800700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 0x7f96800b4dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9690005cd0 tx=0x7f9690009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ec0 tx=0x7f968c009920 comp rx=0 tx=0).stop 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 -- 172.21.15.90:0/3306249662 shutdown_connections 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad6116ad0 0x55fad6116f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/3306249662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad61164b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.307+0000 7f96a71f0b80 1 -- 172.21.15.90:0/3306249662 >> 172.21.15.90:0/3306249662 conn(0x55fad6044be0 msgr2=0x55fad6111150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.308+0000 7f96a71f0b80 1 -- 172.21.15.90:0/3306249662 shutdown_connections 2024-06-02T15:55:16.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.308+0000 7f96a71f0b80 1 -- 172.21.15.90:0/3306249662 wait complete. 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.308+0000 7f96a71f0b80 1 -- start start 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.308+0000 7f96a71f0b80 1 Processor -- start 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 0x55fad6122650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad611f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad611ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fad5f6afa0 con 0x55fad611c020 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55fad6005760 con 0x55fad6116ad0 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fad6123460 con 0x55fad611c940 2024-06-02T15:55:16.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f96a71f0b80 10 client.0 _ll_register_callbacks cb 0x55fad600ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.309+0000 7f9699e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 0x55fad6122650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f969a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad611ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad611f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad611f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52170/0 (socket says 172.21.15.90:52170) 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f969a800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad611ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:44954/0 (socket says 172.21.15.90:44954) 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699400700 1 -- 172.21.15.90:0/1768436573 learned_addr learned my addr 172.21.15.90:0/1768436573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 -- 172.21.15.90:0/1768436573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 msgr2=0x55fad611ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad611ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 -- 172.21.15.90:0/1768436573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 msgr2=0x55fad611f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad611f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 -- 172.21.15.90:0/1768436573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad5fb48d0 con 0x55fad6116ad0 2024-06-02T15:55:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9699e00700 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 0x55fad6122650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f968c009e90 tx=0x7f968c003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2839dca43ca25ff7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.310+0000 7f9673e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c019040 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f968c0159e0 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968c007dc0 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f968c015be0 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f96a71f0b80 10 client.26729 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.311+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fad6035060 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f968c007dc0 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f968c026020 con 0x55fad6116ad0 2024-06-02T15:55:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 20 client.26729 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad612fe00 0x55fad61321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 4 client.26729 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad6076d90 con 0x55fad612fe00 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.312+0000 7f9699400700 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad612fe00 0x55fad61321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.313+0000 7f9699400700 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad612fe00 0x55fad61321f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb9f7782d67a435 server_cookie=3e1de4d7c0bf7174 in_seq=0 out_seq=0 2024-06-02T15:55:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.313+0000 7f9673e00700 10 client.26729 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f9673e00700 1 -- 172.21.15.90:0/1768436573 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fad6076d90 con 0x55fad612fe00 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f9673e00700 10 client.26729 handle_command_reply: tid=0 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 client.26729 shutdown 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 msgr2=0x55fad6122650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 0x55fad6122650 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f968c009e90 tx=0x7f968c003cb0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 shutdown_connections 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad612fe00 0x55fad61321f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad611c940 0x55fad611ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad6116ad0 0x55fad6122650 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/1768436573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad611f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 >> 172.21.15.90:0/1768436573 conn(0x55fad6044be0 msgr2=0x55fad61154a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f9673e00700 0 client.26729 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 shutdown_connections 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.314+0000 7f96a71f0b80 1 -- 172.21.15.90:0/1768436573 wait complete. 2024-06-02T15:55:16.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.315+0000 7f96a71f0b80 20 client.26729 trim_cache size 0 max 16384 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 Processor -- start 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 -- start start 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad61164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96941af990 con 0x55fad611c940 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96941ad160 con 0x55fad611c020 2024-06-02T15:55:16.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.331+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aafb0 con 0x55fad6116ad0 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52174/0 (socket says 172.21.15.90:52174) 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 -- 172.21.15.90:0/4160861703 learned_addr learned my addr 172.21.15.90:0/4160861703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f9699e00700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad61164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f9699400700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 -- 172.21.15.90:0/4160861703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 msgr2=0x55fad61164b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad61164b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 -- 172.21.15.90:0/4160861703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 msgr2=0x55fad611c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f969a800700 1 -- 172.21.15.90:0/4160861703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad611df80 con 0x55fad611c940 2024-06-02T15:55:16.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.332+0000 7f9699400700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.333+0000 7f969a800700 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9690015000 tx=0x7f9690005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea969404dd63850 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.333+0000 7f967ea00700 1 -- 172.21.15.90:0/4160861703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690021040 con 0x55fad611c940 2024-06-02T15:55:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.333+0000 7f967ea00700 1 -- 172.21.15.90:0/4160861703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9690003d30 con 0x55fad611c940 2024-06-02T15:55:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.333+0000 7f967ea00700 1 -- 172.21.15.90:0/4160861703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969000fdf0 con 0x55fad611c940 2024-06-02T15:55:16.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- 172.21.15.90:0/4160861703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 msgr2=0x55fad6110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9690015000 tx=0x7f9690005ec0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- 172.21.15.90:0/4160861703 shutdown_connections 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c940 0x55fad6110c30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad61164b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/4160861703 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c020 0x55fad611c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- 172.21.15.90:0/4160861703 >> 172.21.15.90:0/4160861703 conn(0x55fad6044be0 msgr2=0x55fad610afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- 172.21.15.90:0/4160861703 shutdown_connections 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- 172.21.15.90:0/4160861703 wait complete. 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.334+0000 7f96a71f0b80 1 -- start start 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 Processor -- start 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad6128980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad6128ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96941ab620 con 0x55fad611c020 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96941aca90 con 0x55fad611c940 2024-06-02T15:55:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96941abcf0 con 0x55fad6116ad0 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.335+0000 7f96a71f0b80 10 client.0 _ll_register_callbacks cb 0x55fad600ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f969a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f969a800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:51152/0 (socket says 172.21.15.90:51152) 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad6128ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f969a800700 1 -- 172.21.15.90:0/2990652536 learned_addr learned my addr 172.21.15.90:0/2990652536 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:16.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699e00700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad6128980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 -- 172.21.15.90:0/2990652536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 msgr2=0x55fad6128980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad6128980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 -- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 msgr2=0x55fad6136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 -- 172.21.15.90:0/2990652536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad6137790 con 0x55fad611c020 2024-06-02T15:55:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f969a800700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9699400700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad6128ec0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96840a5740 tx=0x7f96840e5ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07eb6c89c2cd5cf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.336+0000 7f9673e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968401c040 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f96840dfcb0 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9690003d30 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f96840a8e30 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f96a71f0b80 10 client.16434 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fad619a3f0 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.337+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9690003d30 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f96840910b0 con 0x55fad611c020 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 10 client.16434 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 20 client.16434 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:16.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad619ab60 0x55fad619cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 4 client.16434 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fad6070570 con 0x55fad619ab60 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.338+0000 7f9699e00700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad619ab60 0x55fad619cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f9699e00700 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad619ab60 0x55fad619cf80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a099571c63e93404 server_cookie=a327562dca7ae632 in_seq=0 out_seq=0 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f9673e00700 10 client.16434 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f9673e00700 1 -- 172.21.15.90:0/2990652536 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fad6070570 con 0x55fad619ab60 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f9673e00700 10 client.16434 handle_command_reply: tid=0 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 client.16434 shutdown 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 msgr2=0x55fad6128ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad6128ec0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f96840a5740 tx=0x7f96840e5ef0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 shutdown_connections 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55fad619ab60 0x55fad619cf80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55fad611c940 0x55fad6136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad6116ad0 0x55fad6128980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 --2- 172.21.15.90:0/2990652536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fad611c020 0x55fad6128ec0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 >> 172.21.15.90:0/2990652536 conn(0x55fad6044be0 msgr2=0x55fad6110c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f9673e00700 0 client.16434 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 shutdown_connections 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.339+0000 7f96a71f0b80 1 -- 172.21.15.90:0/2990652536 wait complete. 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.340+0000 7f96a71f0b80 20 client.16434 trim_cache size 0 max 16384 2024-06-02T15:55:16.339 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.340+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 msgr2=0x7f96800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.340+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 0x7f96800b4dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9690005cd0 tx=0x7f9690009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.340+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 msgr2=0x7f969419c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.340+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f968400e9f0 tx=0x7f968400ed00 comp rx=0 tx=0).stop 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 shutdown_connections 2024-06-02T15:55:16.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f96800b2900 0x7f96800b4dc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f969410fd10 0x7f96941a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9694075910 0x7f969419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 --2- 172.21.15.90:0/3427424079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9694076230 0x7f969419c2b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 >> 172.21.15.90:0/3427424079 conn(0x7f96940fefc0 msgr2=0x7f96940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 shutdown_connections 2024-06-02T15:55:16.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:16.341+0000 7f969b200700 1 -- 172.21.15.90:0/3427424079 wait complete. 2024-06-02T15:55:17.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:17 smithi120 ceph-mon[36451]: pgmap v843: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:17.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:17 smithi135 ceph-mon[36839]: pgmap v843: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:17 smithi090 ceph-mon[31595]: pgmap v843: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:18.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:55:18.300 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:55:18.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:18 smithi090 ceph-mon[31595]: pgmap v844: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:18 smithi120 ceph-mon[36451]: pgmap v844: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:18.814 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:18 smithi135 ceph-mon[36839]: pgmap v844: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:55:21.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:21 smithi120 ceph-mon[36451]: pgmap v845: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:55:21.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:21 smithi135 ceph-mon[36839]: pgmap v845: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:55:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:21 smithi090 ceph-mon[31595]: pgmap v845: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:55:22.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 -- 172.21.15.90:0/3921764406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81037f0 msgr2=0x7f4ec8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 --2- 172.21.15.90:0/3921764406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81037f0 0x7f4ec8103bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80099c0 tx=0x7f4eb8009cd0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 -- 172.21.15.90:0/3921764406 shutdown_connections 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 --2- 172.21.15.90:0/3921764406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81088f0 0x7f4ec8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 --2- 172.21.15.90:0/3921764406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81037f0 0x7f4ec8103bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 --2- 172.21.15.90:0/3921764406 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81041a0 0x7f4ec81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 -- 172.21.15.90:0/3921764406 >> 172.21.15.90:0/3921764406 conn(0x7f4ec807a6a0 msgr2=0x7f4ec807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 -- 172.21.15.90:0/3921764406 shutdown_connections 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.953+0000 7f4ece200700 1 -- 172.21.15.90:0/3921764406 wait complete. 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.954+0000 7f4ece200700 1 Processor -- start 2024-06-02T15:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.954+0000 7f4ece200700 1 -- start start 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 0x7f4ec819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 0x7f4ec81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8113730 con 0x7f4ec81041a0 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81135b0 con 0x7f4ec81088f0 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ece200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81138b0 con 0x7f4ec81037f0 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 0x7f4ec819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35514/0 (socket says 172.21.15.90:35514) 2024-06-02T15:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 0x7f4ec819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52012/0 (socket says 172.21.15.90:52012) 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 -- 172.21.15.90:0/1028554362 learned_addr learned my addr 172.21.15.90:0/1028554362 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 0x7f4ec81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 -- 172.21.15.90:0/1028554362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 msgr2=0x7f4ec819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 0x7f4ec819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 -- 172.21.15.90:0/1028554362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 msgr2=0x7f4ec81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 0x7f4ec81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ebea00700 1 -- 172.21.15.90:0/1028554362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec819a130 con 0x7f4ec81041a0 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.955+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 0x7f4ec81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ebea00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000e9a0 tx=0x7f4eb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ec4c00700 1 -- 172.21.15.90:0/1028554362 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000ca10 con 0x7f4ec81041a0 2024-06-02T15:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ec4c00700 1 -- 172.21.15.90:0/1028554362 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb000cbb0 con 0x7f4ec81041a0 2024-06-02T15:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec819a420 con 0x7f4ec81041a0 2024-06-02T15:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ec4c00700 1 -- 172.21.15.90:0/1028554362 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0018680 con 0x7f4ec81041a0 2024-06-02T15:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.956+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a66e0 con 0x7f4ec81041a0 2024-06-02T15:55:22.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.962+0000 7f4ec4c00700 1 -- 172.21.15.90:0/1028554362 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4eb0010430 con 0x7f4ec81041a0 2024-06-02T15:55:22.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.962+0000 7f4ed8c3db80 1 Processor -- start 2024-06-02T15:55:22.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 -- start start 2024-06-02T15:55:22.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de21f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de22910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd5dd77ec0 con 0x55bd5de1d8d0 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd5dd67a60 con 0x55bd5de1e1f0 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.963+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd5dd6baa0 con 0x55bd5de224b0 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de1dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35522/0 (socket says 172.21.15.90:35522) 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7e00700 1 -- 172.21.15.90:0/2003807021 learned_addr learned my addr 172.21.15.90:0/2003807021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec4c00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 0x7f4eac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ebea00700 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 -- 172.21.15.90:0/2003807021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 msgr2=0x55bd5de22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 -- 172.21.15.90:0/2003807021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 msgr2=0x55bd5de1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.964+0000 7f4ec7400700 1 -- 172.21.15.90:0/2003807021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd5dd304a0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ec4c00700 1 -- 172.21.15.90:0/1028554362 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4eb00e2770 con 0x7f4ec81041a0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 0x7f4eac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de21f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4ec819ab10 tx=0x7f4eb80036a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=542f84a1269c167d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ebcc00700 1 -- 172.21.15.90:0/2003807021 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8028070 con 0x55bd5de1e1f0 2024-06-02T15:55:22.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ebcc00700 1 -- 172.21.15.90:0/2003807021 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb8009590 con 0x55bd5de1e1f0 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.965+0000 7f4ebcc00700 1 -- 172.21.15.90:0/2003807021 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800b9f0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2003807021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 msgr2=0x55bd5de21f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 0x7f4eac0b4dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009fd0 tx=0x7f4ec0004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de21f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4ec819ab10 tx=0x7f4eb80036a0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2003807021 shutdown_connections 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2003807021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de21f70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2003807021 >> 172.21.15.90:0/2003807021 conn(0x55bd5dd47c60 msgr2=0x55bd5de14090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:22.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2003807021 shutdown_connections 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.966+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2003807021 wait complete. 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.967+0000 7f4ed8c3db80 1 -- start start 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.967+0000 7f4ed8c3db80 1 Processor -- start 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de26cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd5dc6e020 con 0x55bd5de1d8d0 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55bd5dd087e0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bd5de2a680 con 0x55bd5de224b0 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ed8c3db80 10 client.0 _ll_register_callbacks cb 0x55bd5dd11cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:22.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35538/0 (socket says 172.21.15.90:35538) 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ec7400700 1 -- 172.21.15.90:0/971862506 learned_addr learned my addr 172.21.15.90:0/971862506 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ebea00700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de26cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.968+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 -- 172.21.15.90:0/971862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 msgr2=0x55bd5de27200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de27200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 -- 172.21.15.90:0/971862506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 msgr2=0x55bd5de29920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 -- 172.21.15.90:0/971862506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd5dcb7950 con 0x55bd5de1e1f0 2024-06-02T15:55:22.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ec7400700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:22.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ebea00700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de26cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a9910 tx=0x7f4eb00a9ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6c26f839378ee3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:22.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4eb5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:22.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00a5110 con 0x55bd5de1e1f0 2024-06-02T15:55:22.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4eb00cc0b0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.969+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb8009590 con 0x55bd5de1e1f0 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4eb00a9490 con 0x55bd5de1e1f0 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 10 client.26747 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bd5dd380e0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb8009590 con 0x55bd5de1e1f0 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f4eb00e4bb0 con 0x55bd5de1e1f0 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 10 client.26747 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 10 client.26747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 20 client.26747 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5de37120 0x55bd5de39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 4 client.26747 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.970+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd5dd79e10 con 0x55bd5de37120 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.971+0000 7f4ec7400700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5de37120 0x55bd5de39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.971+0000 7f4ec7400700 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5de37120 0x55bd5de39570 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e535476a8511939 server_cookie=74fcec613e13f8ee in_seq=0 out_seq=0 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.971+0000 7f4eb5600700 10 client.26747 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4eb5600700 1 -- 172.21.15.90:0/971862506 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bd5dd79e10 con 0x55bd5de37120 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4eb5600700 10 client.26747 handle_command_reply: tid=0 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 client.26747 shutdown 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 msgr2=0x55bd5de26cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de26cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00a9910 tx=0x7f4eb00a9ec0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 shutdown_connections 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5de37120 0x55bd5de39570 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de27200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de29920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4eb5600700 0 client.26747 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/971862506 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de26cc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 >> 172.21.15.90:0/971862506 conn(0x55bd5dd47c60 msgr2=0x55bd5de182d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 shutdown_connections 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.972+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/971862506 wait complete. 2024-06-02T15:55:22.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.973+0000 7f4ed8c3db80 20 client.26747 trim_cache size 0 max 16384 2024-06-02T15:55:22.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.995+0000 7f4ed8c3db80 1 Processor -- start 2024-06-02T15:55:22.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.995+0000 7f4ed8c3db80 1 -- start start 2024-06-02T15:55:22.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.995+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.995+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de25c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81afa70 con 0x55bd5de1d8d0 2024-06-02T15:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ad240 con 0x55bd5de1e1f0 2024-06-02T15:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ab0b0 con 0x55bd5de224b0 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35548/0 (socket says 172.21.15.90:35548) 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 -- 172.21.15.90:0/2619494315 learned_addr learned my addr 172.21.15.90:0/2619494315 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ebea00700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de25c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 -- 172.21.15.90:0/2619494315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 msgr2=0x55bd5de189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 -- 172.21.15.90:0/2619494315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 msgr2=0x55bd5de25c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de25c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.996+0000 7f4ec7400700 1 -- 172.21.15.90:0/2619494315 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd5de1e020 con 0x55bd5de1d8d0 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.997+0000 7f4ebea00700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de25c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.997+0000 7f4ec7400700 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8004fd0 tx=0x7f4eb801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78b74338ac94e8f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.997+0000 7f4eb7e00700 1 -- 172.21.15.90:0/2619494315 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8028040 con 0x55bd5de1d8d0 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.997+0000 7f4eb7e00700 1 -- 172.21.15.90:0/2619494315 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb8029040 con 0x55bd5de1d8d0 2024-06-02T15:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.997+0000 7f4eb7e00700 1 -- 172.21.15.90:0/2619494315 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800bcc0 con 0x55bd5de1d8d0 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2619494315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 msgr2=0x55bd5de249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8004fd0 tx=0x7f4eb801a040 comp rx=0 tx=0).stop 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2619494315 shutdown_connections 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5de189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1d8d0 0x55bd5de249d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/2619494315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1e1f0 0x55bd5de25c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2619494315 >> 172.21.15.90:0/2619494315 conn(0x55bd5dd47c60 msgr2=0x55bd5de182d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2619494315 shutdown_connections 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.998+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/2619494315 wait complete. 2024-06-02T15:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.999+0000 7f4ed8c3db80 1 -- start start 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.999+0000 7f4ed8c3db80 1 Processor -- start 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.999+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 0x55bd5de2ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.999+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:22.999+0000 7f4ed8c3db80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5dea1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ab700 con 0x55bd5de1e1f0 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81acb70 con 0x55bd5de1d8d0 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ed8c3db80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81abdd0 con 0x55bd5de224b0 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ed8c3db80 10 client.0 _ll_register_callbacks cb 0x55bd5dd11cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ebea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ebea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35552/0 (socket says 172.21.15.90:35552) 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ebea00700 1 -- 172.21.15.90:0/681997085 learned_addr learned my addr 172.21.15.90:0/681997085 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 0x55bd5de2ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:22.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7e00700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5dea1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 -- 172.21.15.90:0/681997085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 msgr2=0x55bd5dea1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5dea1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 -- 172.21.15.90:0/681997085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 msgr2=0x55bd5de2cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ec7400700 1 -- 172.21.15.90:0/681997085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd5dea22a0 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.000+0000 7f4ebea00700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4ec7400700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 0x55bd5de2ca30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b580 tx=0x7f4eb8005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c748db2de35fac5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4eb5600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8046060 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb8029040 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4eb800bb40 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4eb804d050 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4ed8c3db80 10 client.26753 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.001+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bd5dea3460 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb8029040 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f4eb80333f0 con 0x55bd5de1d8d0 2024-06-02T15:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 10 client.26753 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 20 client.26753 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5dea3690 0x55bd5dea5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 4 client.26753 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bd5dd735f0 con 0x55bd5dea3690 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.002+0000 7f4ebea00700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5dea3690 0x55bd5dea5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.003+0000 7f4ebea00700 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5dea3690 0x55bd5dea5a80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e3089e170aa4ee server_cookie=ad19219e657a9210 in_seq=0 out_seq=0 2024-06-02T15:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.003+0000 7f4eb5600700 10 client.26753 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.004+0000 7f4eb5600700 1 -- 172.21.15.90:0/681997085 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bd5dd735f0 con 0x55bd5dea3690 2024-06-02T15:55:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.004+0000 7f4eb5600700 10 client.26753 handle_command_reply: tid=0 2024-06-02T15:55:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.004+0000 7f4ed8c3db80 1 client.26753 shutdown 2024-06-02T15:55:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.004+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 msgr2=0x55bd5de2ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.004+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 0x55bd5de2ca30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800b580 tx=0x7f4eb8005bc0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 shutdown_connections 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55bd5dea3690 0x55bd5dea5a80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bd5de224b0 0x55bd5dea1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55bd5de1d8d0 0x55bd5de2ca30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 --2- 172.21.15.90:0/681997085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd5de1e1f0 0x55bd5de2cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4eb5600700 0 client.26753 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 >> 172.21.15.90:0/681997085 conn(0x55bd5dd47c60 msgr2=0x55bd5de18b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 shutdown_connections 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 1 -- 172.21.15.90:0/681997085 wait complete. 2024-06-02T15:55:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.005+0000 7f4ed8c3db80 20 client.26753 trim_cache size 0 max 16384 2024-06-02T15:55:23.005 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:55:23.005 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:55:23.005 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "4025c8f7-b8dd-4c3b-832a-b6f34f4600d8", 2024-06-02T15:55:23.005 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:55:23.005 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 msgr2=0x7f4eac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 0x7f4eac0b4dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009fd0 tx=0x7f4ec0004ad0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 msgr2=0x7f4ec819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000e9a0 tx=0x7f4eb000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 shutdown_connections 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f4eac0b2900 0x7f4eac0b4dc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec81088f0 0x7f4ec81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ec81037f0 0x7f4ec819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 --2- 172.21.15.90:0/1028554362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ec81041a0 0x7f4ec819fd50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 >> 172.21.15.90:0/1028554362 conn(0x7f4ec807a6a0 msgr2=0x7f4ec8101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 shutdown_connections 2024-06-02T15:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:23.006+0000 7f4ece200700 1 -- 172.21.15.90:0/1028554362 wait complete. 2024-06-02T15:55:23.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:23 smithi120 ceph-mon[36451]: pgmap v846: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:23.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:23 smithi135 ceph-mon[36839]: pgmap v846: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:23 smithi090 ceph-mon[31595]: pgmap v846: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:55:24.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:55:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:24 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:55:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:24 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:24 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:55:24.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:24 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:24 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:55:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:24 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:24 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:55:24.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:24 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:24 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:55:24.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:24 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:24 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:55:24.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:24 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:55:24.737 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:25.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:25 smithi090 ceph-mon[31595]: pgmap v847: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:55:25.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:25 smithi120 ceph-mon[36451]: pgmap v847: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:55:25.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:25 smithi135 ceph-mon[36839]: pgmap v847: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:55:26.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:26 smithi135 ceph-mon[36839]: pgmap v848: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:55:26.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:26 smithi090 ceph-mon[31595]: pgmap v848: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:55:26.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:26 smithi120 ceph-mon[36451]: pgmap v848: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:55:27.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:27 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:55:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:27 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:55:27.708 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:27 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:55:28.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:28 smithi090 ceph-mon[31595]: pgmap v849: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:55:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 -- 172.21.15.90:0/4223857907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc408106030 msgr2=0x7fc408112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 --2- 172.21.15.90:0/4223857907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc408106030 0x7fc408112540 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc40000b0a0 tx=0x7fc40000b3b0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 -- 172.21.15.90:0/4223857907 shutdown_connections 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 --2- 172.21.15.90:0/4223857907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc408106030 0x7fc408112540 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 --2- 172.21.15.90:0/4223857907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc40810b670 0x7fc40810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 --2- 172.21.15.90:0/4223857907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408105670 0x7fc408105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 -- 172.21.15.90:0/4223857907 >> 172.21.15.90:0/4223857907 conn(0x7fc408101160 msgr2=0x7fc408103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 -- 172.21.15.90:0/4223857907 shutdown_connections 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.677+0000 7fc40f000700 1 -- 172.21.15.90:0/4223857907 wait complete. 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.678+0000 7fc40f000700 1 Processor -- start 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 -- start start 2024-06-02T15:55:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408105670 0x7fc4081a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 0x7fc40819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc408115900 con 0x7fc408106030 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc408115780 con 0x7fc40810b670 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40f000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc408115a80 con 0x7fc408105670 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35566/0 (socket says 172.21.15.90:35566) 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40d200700 1 -- 172.21.15.90:0/1363842252 learned_addr learned my addr 172.21.15.90:0/1363842252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 0x7fc40819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.679+0000 7fc40dc00700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408105670 0x7fc4081a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 -- 172.21.15.90:0/1363842252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408105670 msgr2=0x7fc4081a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408105670 0x7fc4081a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 -- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 msgr2=0x7fc4081a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 -- 172.21.15.90:0/1363842252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819c260 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40d200700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 0x7fc40819bb00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc40000b070 tx=0x7fc400004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc406000700 1 -- 172.21.15.90:0/1363842252 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400021070 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc406000700 1 -- 172.21.15.90:0/1363842252 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400007e30 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc406000700 1 -- 172.21.15.90:0/1363842252 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000045f0 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.680+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819c4f0 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.681+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a8810 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.684+0000 7fc406000700 1 -- 172.21.15.90:0/1363842252 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc400007980 con 0x7fc40810b670 2024-06-02T15:55:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.685+0000 7fc41af85b80 1 Processor -- start 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.685+0000 7fc41af85b80 1 -- start start 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a7284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a728f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a686e40 con 0x55ba8a72e020 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a6769e0 con 0x55ba8a728ad0 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc41af85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a67aa20 con 0x55ba8a72e940 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc406000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 0x7fc3f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35576/0 (socket says 172.21.15.90:35576) 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc406000700 1 -- 172.21.15.90:0/1363842252 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc40000f020 con 0x7fc40810b670 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40d200700 1 -- 172.21.15.90:0/2977968149 learned_addr learned my addr 172.21.15.90:0/2977968149 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a7284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 0x7fc3f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.686+0000 7fc40d200700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a728f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 -- 172.21.15.90:0/2977968149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 msgr2=0x55ba8a728f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a728f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 -- 172.21.15.90:0/2977968149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 msgr2=0x55ba8a72e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 -- 172.21.15.90:0/2977968149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8a63f420 con 0x55ba8a72e940 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40d200700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40e600700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 0x7fc3f00b4d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc4000a6bf0 tx=0x7fc400004120 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a7284b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009cb0 tx=0x7fc3fc0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff3d3c7a1db02a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.687+0000 7fc3faa00700 1 -- 172.21.15.90:0/2977968149 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011ce0 con 0x55ba8a72e940 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc3faa00700 1 -- 172.21.15.90:0/2977968149 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc0044e0 con 0x55ba8a72e940 2024-06-02T15:55:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc3faa00700 1 -- 172.21.15.90:0/2977968149 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc005280 con 0x55ba8a72e940 2024-06-02T15:55:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 -- 172.21.15.90:0/2977968149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 msgr2=0x55ba8a7284b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a7284b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009cb0 tx=0x7fc3fc0058e0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 -- 172.21.15.90:0/2977968149 shutdown_connections 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a728f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a72e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2977968149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a7284b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.688+0000 7fc41af85b80 1 -- 172.21.15.90:0/2977968149 >> 172.21.15.90:0/2977968149 conn(0x55ba8a656be0 msgr2=0x55ba8a723150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.689+0000 7fc41af85b80 1 -- 172.21.15.90:0/2977968149 shutdown_connections 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.689+0000 7fc41af85b80 1 -- 172.21.15.90:0/2977968149 wait complete. 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.689+0000 7fc41af85b80 1 -- start start 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.689+0000 7fc41af85b80 1 Processor -- start 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a734630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a731f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a57cfa0 con 0x55ba8a72e020 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a617760 con 0x55ba8a728ad0 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba8a735460 con 0x55ba8a72e940 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc41af85b80 10 client.0 _ll_register_callbacks cb 0x55ba8a620c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35592/0 (socket says 172.21.15.90:35592) 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc40dc00700 1 -- 172.21.15.90:0/2578013486 learned_addr learned my addr 172.21.15.90:0/2578013486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc40d200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a734630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.690+0000 7fc40d200700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a731f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 -- 172.21.15.90:0/2578013486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 msgr2=0x55ba8a731f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a731f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 -- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 msgr2=0x55ba8a734630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a734630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 -- 172.21.15.90:0/2578013486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8a5c68d0 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc40dc00700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc004df0 tx=0x7fc3fc011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54464dae56141cfc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc3ef400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.691+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc005260 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3fc0044e0 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc3fc005400 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc3fc032050 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc41af85b80 10 client.16464 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ba8a647060 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc0044e0 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.692+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fc3fc018070 con 0x55ba8a72e020 2024-06-02T15:55:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 20 client.16464 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a741f20 0x55ba8a744310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 4 client.16464 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba8a688d90 con 0x55ba8a741f20 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.693+0000 7fc40d200700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a741f20 0x55ba8a744310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.694+0000 7fc40d200700 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a741f20 0x55ba8a744310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54436365d7594a84 server_cookie=1adb6da60c8a012f in_seq=0 out_seq=0 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.694+0000 7fc3ef400700 10 client.16464 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.694+0000 7fc3ef400700 1 -- 172.21.15.90:0/2578013486 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ba8a688d90 con 0x55ba8a741f20 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.694+0000 7fc3ef400700 10 client.16464 handle_command_reply: tid=0 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.694+0000 7fc41af85b80 1 client.16464 shutdown 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 msgr2=0x55ba8a7319d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc004df0 tx=0x7fc3fc011d80 comp rx=0 tx=0).stop 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 shutdown_connections 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a741f20 0x55ba8a744310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a731f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a734630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc3ef400700 0 client.16464 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 --2- 172.21.15.90:0/2578013486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7319d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 >> 172.21.15.90:0/2578013486 conn(0x55ba8a656be0 msgr2=0x55ba8a7274a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 shutdown_connections 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 1 -- 172.21.15.90:0/2578013486 wait complete. 2024-06-02T15:55:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.695+0000 7fc41af85b80 20 client.16464 trim_cache size 0 max 16384 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.718+0000 7fc41af85b80 1 Processor -- start 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.718+0000 7fc41af85b80 1 -- start start 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.718+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.718+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.718+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a72baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc41af85b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081b3bc0 con 0x55ba8a72e020 2024-06-02T15:55:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc41af85b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081b1390 con 0x55ba8a728ad0 2024-06-02T15:55:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc41af85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081af1e0 con 0x55ba8a72e940 2024-06-02T15:55:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35598/0 (socket says 172.21.15.90:35598) 2024-06-02T15:55:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40dc00700 1 -- 172.21.15.90:0/811944141 learned_addr learned my addr 172.21.15.90:0/811944141 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:28.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40d200700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40d200700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a72baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40dc00700 1 -- 172.21.15.90:0/811944141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 msgr2=0x55ba8a72baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.719+0000 7fc40dc00700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a72baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc40dc00700 1 -- 172.21.15.90:0/811944141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 msgr2=0x55ba8a7284b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc40dc00700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7284b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc40dc00700 1 -- 172.21.15.90:0/811944141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8a72ff40 con 0x55ba8a72e020 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc40d200700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc40dc00700 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1994d22c8c91e241 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc3fa000700 1 -- 172.21.15.90:0/811944141 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011720 con 0x55ba8a72e020 2024-06-02T15:55:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc3fa000700 1 -- 172.21.15.90:0/811944141 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc029070 con 0x55ba8a72e020 2024-06-02T15:55:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.720+0000 7fc3fa000700 1 -- 172.21.15.90:0/811944141 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc005060 con 0x55ba8a72e020 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 -- 172.21.15.90:0/811944141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 msgr2=0x55ba8a7248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc000c00 tx=0x7fc3fc009870 comp rx=0 tx=0).stop 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 -- 172.21.15.90:0/811944141 shutdown_connections 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e940 0x55ba8a72baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 --2- 172.21.15.90:0/811944141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e020 0x55ba8a7248b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 -- 172.21.15.90:0/811944141 >> 172.21.15.90:0/811944141 conn(0x55ba8a656be0 msgr2=0x55ba8a71cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 -- 172.21.15.90:0/811944141 shutdown_connections 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.721+0000 7fc41af85b80 1 -- 172.21.15.90:0/811944141 wait complete. 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.722+0000 7fc41af85b80 1 -- start start 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.722+0000 7fc41af85b80 1 Processor -- start 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.722+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.722+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.722+0000 7fc41af85b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 0x55ba8a749f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc41af85b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081af850 con 0x55ba8a72e940 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc41af85b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081b0cc0 con 0x55ba8a728ad0 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc41af85b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aff20 con 0x55ba8a72e020 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc41af85b80 10 client.0 _ll_register_callbacks cb 0x55ba8a620c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 0x55ba8a749f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:52114/0 (socket says 172.21.15.90:52114) 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 -- 172.21.15.90:0/4194069809 learned_addr learned my addr 172.21.15.90:0/4194069809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 0x55ba8a749f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35604/0 (socket says 172.21.15.90:35604) 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40d200700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 -- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 msgr2=0x55ba8a7494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 -- 172.21.15.90:0/4194069809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 msgr2=0x55ba8a749f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.723+0000 7fc40dc00700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 0x55ba8a749f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc40dc00700 1 -- 172.21.15.90:0/4194069809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8a7ab950 con 0x55ba8a72e020 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc40dc00700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009410 tx=0x7fc3fc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e671d040dcfd13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc3ef400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:55:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0045b0 con 0x55ba8a72e020 2024-06-02T15:55:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc3fc019a50 con 0x55ba8a72e020 2024-06-02T15:55:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.724+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3fc029070 con 0x55ba8a72e020 2024-06-02T15:55:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc3fc045050 con 0x55ba8a72e020 2024-06-02T15:55:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 10 client.26707 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ba8a7acb80 con 0x55ba8a72e020 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc029070 con 0x55ba8a72e020 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fc3fc03e060 con 0x55ba8a72e020 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 10 client.26707 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 20 client.26707 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a7ad1f0 0x55ba8a7af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 4 client.26707 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.725+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba8a682570 con 0x55ba8a7ad1f0 2024-06-02T15:55:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc40d200700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a7ad1f0 0x55ba8a7af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc40d200700 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a7ad1f0 0x55ba8a7af610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa49df9206c51211 server_cookie=57fc6c581a69ba49 in_seq=0 out_seq=0 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc3ef400700 10 client.26707 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc3ef400700 1 -- 172.21.15.90:0/4194069809 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba8a682570 con 0x55ba8a7ad1f0 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc3ef400700 10 client.26707 handle_command_reply: tid=0 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.726+0000 7fc41af85b80 1 client.26707 shutdown 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 msgr2=0x55ba8a749a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009410 tx=0x7fc3fc0042f0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 shutdown_connections 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55ba8a7ad1f0 0x55ba8a7af610 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba8a72e940 0x55ba8a749f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba8a728ad0 0x55ba8a7494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc3ef400700 0 client.26707 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 --2- 172.21.15.90:0/4194069809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba8a72e020 0x55ba8a749a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 >> 172.21.15.90:0/4194069809 conn(0x55ba8a656be0 msgr2=0x55ba8a722c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 shutdown_connections 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 1 -- 172.21.15.90:0/4194069809 wait complete. 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.727+0000 7fc41af85b80 20 client.26707 trim_cache size 0 max 16384 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:55:28.729 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:55:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 msgr2=0x7fc3f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 0x7fc3f00b4d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc4000a6bf0 tx=0x7fc400004120 comp rx=0 tx=0).stop 2024-06-02T15:55:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 msgr2=0x7fc40819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 0x7fc40819bb00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc40000b070 tx=0x7fc400004c80 comp rx=0 tx=0).stop 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 shutdown_connections 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fc3f00b2880 0x7fc3f00b4d40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc40810b670 0x7fc40819bb00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408105670 0x7fc4081a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 --2- 172.21.15.90:0/1363842252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408106030 0x7fc4081a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 >> 172.21.15.90:0/1363842252 conn(0x7fc408101160 msgr2=0x7fc408101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 shutdown_connections 2024-06-02T15:55:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:28.728+0000 7fc40f000700 1 -- 172.21.15.90:0/1363842252 wait complete. 2024-06-02T15:55:28.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:28 smithi120 ceph-mon[36451]: pgmap v849: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:55:28.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:28 smithi135 ceph-mon[36839]: pgmap v849: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:55:30.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:55:30.668 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:55:31.117 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:31 smithi120 ceph-mon[36451]: pgmap v850: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:31 smithi135 ceph-mon[36839]: pgmap v850: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:31 smithi090 ceph-mon[31595]: pgmap v850: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:55:33.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:33 smithi120 ceph-mon[36451]: pgmap v851: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:33.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:33 smithi135 ceph-mon[36839]: pgmap v851: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:33 smithi090 ceph-mon[31595]: pgmap v851: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:55:34.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:34 smithi090 ceph-mon[31595]: pgmap v852: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:34.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:34 smithi120 ceph-mon[36451]: pgmap v852: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:34.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:34 smithi135 ceph-mon[36839]: pgmap v852: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:55:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.497+0000 7f8de4e00700 1 -- 172.21.15.90:0/2456593429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 msgr2=0x7f8de0075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.497+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2456593429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de0075b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009990 tx=0x7f8dd0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 -- 172.21.15.90:0/2456593429 shutdown_connections 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2456593429 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2456593429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de0075b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2456593429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 0x7f8de010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 -- 172.21.15.90:0/2456593429 >> 172.21.15.90:0/2456593429 conn(0x7f8de007a6a0 msgr2=0x7f8de007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 -- 172.21.15.90:0/2456593429 shutdown_connections 2024-06-02T15:55:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.498+0000 7f8de4e00700 1 -- 172.21.15.90:0/2456593429 wait complete. 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 Processor -- start 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 -- start start 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 0x7f8de019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0115960 con 0x7f8de0075770 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8de01157e0 con 0x7f8de010ff40 2024-06-02T15:55:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8de4e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0115ae0 con 0x7f8de0076090 2024-06-02T15:55:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8ddde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.500+0000 7f8ddde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38984/0 (socket says 172.21.15.90:38984) 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 -- 172.21.15.90:0/2480194402 learned_addr learned my addr 172.21.15.90:0/2480194402 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8dde800700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8dd5400700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 0x7f8de019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 -- 172.21.15.90:0/2480194402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 msgr2=0x7f8de019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 0x7f8de019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 -- 172.21.15.90:0/2480194402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 msgr2=0x7f8de01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 -- 172.21.15.90:0/2480194402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de01a0e00 con 0x7f8de0075770 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8dde800700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.501+0000 7f8ddde00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005910 tx=0x7f8dd000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.502+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0027070 con 0x7f8de0075770 2024-06-02T15:55:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.502+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd0022470 con 0x7f8de0075770 2024-06-02T15:55:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.502+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de01a1090 con 0x7f8de0075770 2024-06-02T15:55:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.502+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0017630 con 0x7f8de0075770 2024-06-02T15:55:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.502+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de01a8970 con 0x7f8de0075770 2024-06-02T15:55:35.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.503+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0076e60 con 0x7f8de0075770 2024-06-02T15:55:35.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.505+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8dd002a070 con 0x7f8de0075770 2024-06-02T15:55:35.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.508+0000 7f8dd7400700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 0x7f8dc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:35.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.508+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8dd00e3760 con 0x7f8de0075770 2024-06-02T15:55:35.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.508+0000 7f8dd5400700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 0x7f8dc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:35.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.509+0000 7f8dd5400700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 0x7f8dc40b4d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8de0111b90 tx=0x7f8dc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:35.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.511+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8dd0013070 con 0x7f8de0075770 2024-06-02T15:55:35.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.795+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8de002e560 con 0x7f8de0075770 2024-06-02T15:55:35.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.797+0000 7f8dd7400700 1 -- 172.21.15.90:0/2480194402 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11889 (secure 0 0 0) 0x7f8dd00ab100 con 0x7f8de0075770 2024-06-02T15:55:35.794 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:55:35.794 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:54:27.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:55:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.800+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 msgr2=0x7f8dc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.800+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 0x7f8dc40b4d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8de0111b90 tx=0x7f8dc8009260 comp rx=0 tx=0).stop 2024-06-02T15:55:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.800+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 msgr2=0x7f8de019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:35.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.800+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005910 tx=0x7f8dd000b670 comp rx=0 tx=0).stop 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 shutdown_connections 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8dc40b2880 0x7f8dc40b4d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8de010ff40 0x7f8de01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8de0075770 0x7f8de019bdf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 --2- 172.21.15.90:0/2480194402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8de0076090 0x7f8de019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 >> 172.21.15.90:0/2480194402 conn(0x7f8de007a6a0 msgr2=0x7f8de01006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:35.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 shutdown_connections 2024-06-02T15:55:35.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:35.801+0000 7f8de4e00700 1 -- 172.21.15.90:0/2480194402 wait complete. 2024-06-02T15:55:35.799 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-06-02T15:55:36.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:35 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2480194402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:36.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:35 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2480194402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:36.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:35 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2480194402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:55:37.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:36 smithi090 ceph-mon[31595]: pgmap v853: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:55:37.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:36 smithi120 ceph-mon[36451]: pgmap v853: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:55:37.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:36 smithi135 ceph-mon[36839]: pgmap v853: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:55:38.579 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:55:39.016 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:39.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:39 smithi090 ceph-mon[31595]: pgmap v854: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:55:39.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:39 smithi120 ceph-mon[36451]: pgmap v854: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:55:39.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:39 smithi135 ceph-mon[36839]: pgmap v854: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:55:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:40 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:55:40.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:40 smithi135 ceph-mon[36839]: pgmap v855: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:40.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:40 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:55:40.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:40 smithi090 ceph-mon[31595]: pgmap v855: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:40.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:40 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:55:40.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:40 smithi120 ceph-mon[36451]: pgmap v855: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:41.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:55:41.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:55:41.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:55:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:55:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:55:41.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:55:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:55:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:55:41.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:55:42.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 -- 172.21.15.90:0/1418853131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38105760 msgr2=0x7f5f38105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/1418853131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38105760 0x7f5f38105be0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009990 tx=0x7f5f24009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 -- 172.21.15.90:0/1418853131 shutdown_connections 2024-06-02T15:55:42.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/1418853131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38106120 0x7f5f38112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/1418853131 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 0x7f5f3810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/1418853131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38105760 0x7f5f38105be0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 -- 172.21.15.90:0/1418853131 >> 172.21.15.90:0/1418853131 conn(0x7f5f38101250 msgr2=0x7f5f38103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 -- 172.21.15.90:0/1418853131 shutdown_connections 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.223+0000 7f5f3ce00700 1 -- 172.21.15.90:0/1418853131 wait complete. 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.224+0000 7f5f3ce00700 1 Processor -- start 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.224+0000 7f5f3ce00700 1 -- start start 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 0x7f5f3819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 0x7f5f381a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381159a0 con 0x7f5f38105760 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38115820 con 0x7f5f3810b760 2024-06-02T15:55:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f3ce00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f38115b20 con 0x7f5f38106120 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 0x7f5f3819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:36780/0 (socket says 172.21.15.90:36780) 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 -- 172.21.15.90:0/3523233552 learned_addr learned my addr 172.21.15.90:0/3523233552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f36800700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 0x7f5f381a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 -- 172.21.15.90:0/3523233552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 msgr2=0x7f5f381a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 0x7f5f381a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 -- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 msgr2=0x7f5f3819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 0x7f5f3819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35400700 1 -- 172.21.15.90:0/3523233552 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f381a0e40 con 0x7f5f38106120 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.225+0000 7f5f35e00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 0x7f5f3819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f35400700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000c00 tx=0x7f5f2400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f2ea00700 1 -- 172.21.15.90:0/3523233552 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24027070 con 0x7f5f38106120 2024-06-02T15:55:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f2ea00700 1 -- 172.21.15.90:0/3523233552 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f24022470 con 0x7f5f38106120 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f381a10d0 con 0x7f5f38106120 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f2ea00700 1 -- 172.21.15.90:0/3523233552 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24017650 con 0x7f5f38106120 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.226+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f381a89b0 con 0x7f5f38106120 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.230+0000 7f5f2ea00700 1 -- 172.21.15.90:0/3523233552 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f2402a070 con 0x7f5f38106120 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 Processor -- start 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 -- start start 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae85d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae857b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5580ae7b5e40 con 0x5580ae858140 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5580ae7a59e0 con 0x5580ae85d610 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f47708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5580ae7a9a20 con 0x5580ae85df30 2024-06-02T15:55:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.231+0000 7f5f2ea00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 0x7f5f1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae857b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39422/0 (socket says 172.21.15.90:39422) 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f36800700 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae85d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35400700 1 -- 172.21.15.90:0/175293154 learned_addr learned my addr 172.21.15.90:0/175293154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f2ea00700 1 -- 172.21.15.90:0/3523233552 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f5f240e3270 con 0x7f5f38106120 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f36800700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 0x7f5f1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 -- 172.21.15.90:0/175293154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 msgr2=0x5580ae85d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae85d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 -- 172.21.15.90:0/175293154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 msgr2=0x5580ae8585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f35e00700 1 -- 172.21.15.90:0/175293154 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ae76e420 con 0x5580ae85df30 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.232+0000 7f5f36800700 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae85d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.233+0000 7f5f35e00700 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae857b20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f280114f0 tx=0x7f5f28009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198d536f348c8898 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.233+0000 7f5f36800700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 0x7f5f1c0b4d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009da0 tx=0x7f5f3000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.233+0000 7f5f23400700 1 -- 172.21.15.90:0/175293154 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28004310 con 0x5580ae85df30 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.233+0000 7f5f23400700 1 -- 172.21.15.90:0/175293154 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f280044b0 con 0x5580ae85df30 2024-06-02T15:55:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.233+0000 7f5f23400700 1 -- 172.21.15.90:0/175293154 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28003680 con 0x5580ae85df30 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 -- 172.21.15.90:0/175293154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 msgr2=0x5580ae857b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae857b20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5f280114f0 tx=0x7f5f28009f90 comp rx=0 tx=0).stop 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 -- 172.21.15.90:0/175293154 shutdown_connections 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae85d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 --2- 172.21.15.90:0/175293154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae857b20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 -- 172.21.15.90:0/175293154 >> 172.21.15.90:0/175293154 conn(0x5580ae785be0 msgr2=0x5580ae852690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 -- 172.21.15.90:0/175293154 shutdown_connections 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.234+0000 7f5f47708b80 1 -- 172.21.15.90:0/175293154 wait complete. 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.235+0000 7f5f47708b80 1 -- start start 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.235+0000 7f5f47708b80 1 Processor -- start 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae863b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae860ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae861420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5580ae6abfa0 con 0x5580ae858140 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5580ae746760 con 0x5580ae85d610 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5580ae8649e0 con 0x5580ae85df30 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f47708b80 10 client.0 _ll_register_callbacks cb 0x5580ae74fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae863b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae863b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39436/0 (socket says 172.21.15.90:39436) 2024-06-02T15:55:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f35e00700 1 -- 172.21.15.90:0/1134470659 learned_addr learned my addr 172.21.15.90:0/1134470659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f35400700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae860ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.236+0000 7f5f36800700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae861420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 -- 172.21.15.90:0/1134470659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 msgr2=0x5580ae861420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae861420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 -- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 msgr2=0x5580ae863b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae863b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 -- 172.21.15.90:0/1134470659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ae6f58d0 con 0x5580ae85d610 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f35400700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae860ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5f240a6590 tx=0x7f5f2406fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25dbd92ff1b10202 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f17e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240aa2e0 con 0x5580ae85d610 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f5f24071110 con 0x5580ae85d610 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.237+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f280044b0 con 0x5580ae85d610 2024-06-02T15:55:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5f240a9c80 con 0x5580ae85d610 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f47708b80 10 client.26789 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5580ae776060 con 0x5580ae85d610 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f280044b0 con 0x5580ae85d610 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f5f2406fb70 con 0x5580ae85d610 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f47708b80 10 client.26789 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.238+0000 7f5f47708b80 10 client.26789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.239+0000 7f5f47708b80 20 client.26789 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.239+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae879c00 0x5580ae87c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.239+0000 7f5f47708b80 4 client.26789 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.239+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580ae7b7d90 con 0x5580ae879c00 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.239+0000 7f5f35e00700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae879c00 0x5580ae87c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.240+0000 7f5f35e00700 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae879c00 0x5580ae87c0a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9aed80cc51924b2 server_cookie=b3e4a6e301812c55 in_seq=0 out_seq=0 2024-06-02T15:55:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.240+0000 7f5f17e00700 10 client.26789 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.240+0000 7f5f17e00700 1 -- 172.21.15.90:0/1134470659 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5580ae7b7d90 con 0x5580ae879c00 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.240+0000 7f5f17e00700 10 client.26789 handle_command_reply: tid=0 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 client.26789 shutdown 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 msgr2=0x5580ae860ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae860ee0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5f240a6590 tx=0x7f5f2406fb00 comp rx=0 tx=0).stop 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 shutdown_connections 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae879c00 0x5580ae87c0a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae861420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae863b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 --2- 172.21.15.90:0/1134470659 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae860ee0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f17e00700 0 client.26789 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 >> 172.21.15.90:0/1134470659 conn(0x5580ae785be0 msgr2=0x5580ae851d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 shutdown_connections 2024-06-02T15:55:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 1 -- 172.21.15.90:0/1134470659 wait complete. 2024-06-02T15:55:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.241+0000 7f5f47708b80 20 client.26789 trim_cache size 0 max 16384 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 Processor -- start 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 -- start start 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae862ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae8819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381af8c0 con 0x5580ae858140 2024-06-02T15:55:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381ad090 con 0x5580ae85d610 2024-06-02T15:55:42.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.258+0000 7f5f47708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381aafb0 con 0x5580ae85df30 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae862ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39450/0 (socket says 172.21.15.90:39450) 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 -- 172.21.15.90:0/3983708002 learned_addr learned my addr 172.21.15.90:0/3983708002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f36800700 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae8819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 -- 172.21.15.90:0/3983708002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 msgr2=0x5580ae8819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae8819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 -- 172.21.15.90:0/3983708002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 msgr2=0x5580ae862ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae862ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.259+0000 7f5f35e00700 1 -- 172.21.15.90:0/3983708002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ae87c720 con 0x5580ae858140 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.260+0000 7f5f35e00700 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005490 tx=0x7f5f28005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9469e9791951fb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.260+0000 7f5f22a00700 1 -- 172.21.15.90:0/3983708002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28032070 con 0x5580ae858140 2024-06-02T15:55:42.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.260+0000 7f5f22a00700 1 -- 172.21.15.90:0/3983708002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2802a070 con 0x5580ae858140 2024-06-02T15:55:42.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.260+0000 7f5f22a00700 1 -- 172.21.15.90:0/3983708002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f28003c20 con 0x5580ae858140 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- 172.21.15.90:0/3983708002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 msgr2=0x5580ae8524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005490 tx=0x7f5f28005420 comp rx=0 tx=0).stop 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- 172.21.15.90:0/3983708002 shutdown_connections 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae85df30 0x5580ae8819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae858140 0x5580ae8524f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 --2- 172.21.15.90:0/3983708002 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85d610 0x5580ae862ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- 172.21.15.90:0/3983708002 >> 172.21.15.90:0/3983708002 conn(0x5580ae785be0 msgr2=0x5580ae862590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- 172.21.15.90:0/3983708002 shutdown_connections 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- 172.21.15.90:0/3983708002 wait complete. 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.261+0000 7f5f47708b80 1 -- start start 2024-06-02T15:55:42.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 Processor -- start 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 0x5580ae872530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85df30 0x5580ae86f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381ab550 con 0x5580ae85d610 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381ac9c0 con 0x5580ae85df30 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f381abc20 con 0x5580ae858140 2024-06-02T15:55:42.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f47708b80 10 client.0 _ll_register_callbacks cb 0x5580ae74fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:42.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.262+0000 7f5f35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39458/0 (socket says 172.21.15.90:39458) 2024-06-02T15:55:42.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 -- 172.21.15.90:0/772494504 learned_addr learned my addr 172.21.15.90:0/772494504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:42.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35e00700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 0x5580ae872530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f36800700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85df30 0x5580ae86f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 -- 172.21.15.90:0/772494504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 msgr2=0x5580ae872530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 0x5580ae872530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 -- 172.21.15.90:0/772494504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85df30 msgr2=0x5580ae86f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85df30 0x5580ae86f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 -- 172.21.15.90:0/772494504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ae86fa40 con 0x5580ae85d610 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35e00700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 0x5580ae872530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f35400700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f240aa4a0 tx=0x7f5f240e3170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea47a13460614e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f17e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.263+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a9e60 con 0x5580ae85d610 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f5f240b3060 con 0x5580ae85d610 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2802a070 con 0x5580ae85d610 2024-06-02T15:55:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5f240aa020 con 0x5580ae85d610 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 10 client.16494 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5580ae8e4f20 con 0x5580ae85d610 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2802a070 con 0x5580ae85d610 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f5f240a5e80 con 0x5580ae85d610 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 10 client.16494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 20 client.16494 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae8e5520 0x5580ae8e7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 4 client.16494 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.264+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580ae7b1570 con 0x5580ae8e5520 2024-06-02T15:55:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.265+0000 7f5f35e00700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae8e5520 0x5580ae8e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:42.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.265+0000 7f5f35e00700 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae8e5520 0x5580ae8e7940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e72c1e32664ee27 server_cookie=bb7a3a633d3898c2 in_seq=0 out_seq=0 2024-06-02T15:55:42.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.265+0000 7f5f17e00700 10 client.16494 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:42.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f17e00700 1 -- 172.21.15.90:0/772494504 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580ae7b1570 con 0x5580ae8e5520 2024-06-02T15:55:42.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f17e00700 10 client.16494 handle_command_reply: tid=0 2024-06-02T15:55:42.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 client.16494 shutdown 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 msgr2=0x5580ae872a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5f240aa4a0 tx=0x7f5f240e3170 comp rx=0 tx=0).stop 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 shutdown_connections 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x5580ae8e5520 0x5580ae8e7940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5580ae85df30 0x5580ae86f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5580ae858140 0x5580ae872530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 --2- 172.21.15.90:0/772494504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580ae85d610 0x5580ae872a70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f17e00700 0 client.16494 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 >> 172.21.15.90:0/772494504 conn(0x5580ae785be0 msgr2=0x5580ae85ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 shutdown_connections 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 1 -- 172.21.15.90:0/772494504 wait complete. 2024-06-02T15:55:42.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.266+0000 7f5f47708b80 20 client.16494 trim_cache size 0 max 16384 2024-06-02T15:55:42.267 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 msgr2=0x7f5f1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 0x7f5f1c0b4d40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009da0 tx=0x7f5f3000b040 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 msgr2=0x7f5f3819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000c00 tx=0x7f5f2400bfd0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 shutdown_connections 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f5f1c0b2880 0x7f5f1c0b4d40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5f3810b760 0x7f5f381a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f38105760 0x7f5f3819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 --2- 172.21.15.90:0/3523233552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f38106120 0x7f5f3819c370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 >> 172.21.15.90:0/3523233552 conn(0x7f5f38101250 msgr2=0x7f5f38101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 shutdown_connections 2024-06-02T15:55:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:42.267+0000 7f5f3ce00700 1 -- 172.21.15.90:0/3523233552 wait complete. 2024-06-02T15:55:42.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:42 smithi090 ceph-mon[31595]: pgmap v856: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:42.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:42 smithi120 ceph-mon[36451]: pgmap v856: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:42 smithi135 ceph-mon[36839]: pgmap v856: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:43.488 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:55:43.943 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:45.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:45 smithi120 ceph-mon[36451]: pgmap v857: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:45.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:45 smithi135 ceph-mon[36839]: pgmap v857: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:45 smithi090 ceph-mon[31595]: pgmap v857: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:46.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:46 smithi120 ceph-mon[36451]: pgmap v858: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:46.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:46 smithi135 ceph-mon[36839]: pgmap v858: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:46.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:46 smithi090 ceph-mon[31595]: pgmap v858: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 -- 172.21.15.90:0/3516884121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 msgr2=0x7f34bc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 --2- 172.21.15.90:0/3516884121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc06c090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009990 tx=0x7f34a4009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 -- 172.21.15.90:0/3516884121 shutdown_connections 2024-06-02T15:55:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 --2- 172.21.15.90:0/3516884121 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 --2- 172.21.15.90:0/3516884121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 --2- 172.21.15.90:0/3516884121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc06c090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.237+0000 7f34c1a00700 1 -- 172.21.15.90:0/3516884121 >> 172.21.15.90:0/3516884121 conn(0x7f34bc0793f0 msgr2=0x7f34bc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.238+0000 7f34c1a00700 1 -- 172.21.15.90:0/3516884121 shutdown_connections 2024-06-02T15:55:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.238+0000 7f34c1a00700 1 -- 172.21.15.90:0/3516884121 wait complete. 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 Processor -- start 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 -- start start 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1136e0 con 0x7f34bc06bc10 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc113560 con 0x7f34bc06c5d0 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.239+0000 7f34c1a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc113860 con 0x7f34bc1095d0 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34b2000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39462/0 (socket says 172.21.15.90:39462) 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 -- 172.21.15.90:0/2226342240 learned_addr learned my addr 172.21.15.90:0/2226342240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34bb400700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 -- 172.21.15.90:0/2226342240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 msgr2=0x7f34bc103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 -- 172.21.15.90:0/2226342240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 msgr2=0x7f34bc102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34baa00700 1 -- 172.21.15.90:0/2226342240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1a8120 con 0x7f34bc06bc10 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34b2000700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.240+0000 7f34bb400700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34baa00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ef10 tx=0x7f34ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34b3e00700 1 -- 172.21.15.90:0/2226342240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00c9c0 con 0x7f34bc06bc10 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34b3e00700 1 -- 172.21.15.90:0/2226342240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac019070 con 0x7f34bc06bc10 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34b3e00700 1 -- 172.21.15.90:0/2226342240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac014480 con 0x7f34bc06bc10 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1a83b0 con 0x7f34bc06bc10 2024-06-02T15:55:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.241+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a8910 con 0x7f34bc06bc10 2024-06-02T15:55:48.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.245+0000 7f34b3e00700 1 -- 172.21.15.90:0/2226342240 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f34ac012050 con 0x7f34bc06bc10 2024-06-02T15:55:48.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.246+0000 7f34cc3f8b80 1 Processor -- start 2024-06-02T15:55:48.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 -- start start 2024-06-02T15:55:48.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34b3e00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 0x7f34a00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd80d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd80d880 0x5629dd80dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34b3e00700 1 -- 172.21.15.90:0/2226342240 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f34ac0e1fc0 con 0x7f34bc06bc10 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629dd76be40 con 0x5629dd8137d0 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5629dd75b9e0 con 0x5629dd812eb0 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629dd75fa20 con 0x5629dd80d880 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34bb400700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 0x7f34a00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34baa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34baa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:37686/0 (socket says 172.21.15.90:37686) 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 -- 172.21.15.90:0/3944214887 learned_addr learned my addr 172.21.15.90:0/3944214887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.247+0000 7f34b2000700 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd80d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34bb400700 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd80d880 0x5629dd80dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 -- 172.21.15.90:0/3944214887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd80d880 msgr2=0x5629dd80dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd80d880 0x5629dd80dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 -- 172.21.15.90:0/3944214887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 msgr2=0x5629dd80d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd80d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 -- 172.21.15.90:0/3944214887 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dd724420 con 0x5629dd812eb0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34baa00700 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0a52b0 tx=0x7f34ac0a8a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a301ff042415778d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.248+0000 7f34bb400700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 0x7f34a00b4dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f34b40097a0 tx=0x7f34b4009cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.249+0000 7f34abe00700 1 -- 172.21.15.90:0/3944214887 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0a9230 con 0x5629dd812eb0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.249+0000 7f34abe00700 1 -- 172.21.15.90:0/3944214887 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac0ad5e0 con 0x5629dd812eb0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.249+0000 7f34abe00700 1 -- 172.21.15.90:0/3944214887 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0dfe10 con 0x5629dd812eb0 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.249+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3944214887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 msgr2=0x5629dd813290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.249+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0a52b0 tx=0x7f34ac0a8a20 comp rx=0 tx=0).stop 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3944214887 shutdown_connections 2024-06-02T15:55:48.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd80d880 0x5629dd80dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd813290 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3944214887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd80d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3944214887 >> 172.21.15.90:0/3944214887 conn(0x5629dd73bbe0 msgr2=0x5629dd808010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3944214887 shutdown_connections 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3944214887 wait complete. 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.250+0000 7f34cc3f8b80 1 -- start start 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 Processor -- start 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd819490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd812eb0 0x5629dd816830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629dd661fa0 con 0x5629dd8137d0 2024-06-02T15:55:48.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5629dd6fc760 con 0x5629dd80d880 2024-06-02T15:55:48.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5629dd81a240 con 0x5629dd812eb0 2024-06-02T15:55:48.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.251+0000 7f34cc3f8b80 10 client.0 _ll_register_callbacks cb 0x5629dd705c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:48.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39480/0 (socket says 172.21.15.90:39480) 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34b2000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd812eb0 0x5629dd816830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 -- 172.21.15.90:0/1021472685 learned_addr learned my addr 172.21.15.90:0/1021472685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34baa00700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd819490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 -- 172.21.15.90:0/1021472685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd812eb0 msgr2=0x5629dd816830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd812eb0 0x5629dd816830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 -- 172.21.15.90:0/1021472685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 msgr2=0x5629dd819490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd819490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 -- 172.21.15.90:0/1021472685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dd6ab8d0 con 0x5629dd8137d0 2024-06-02T15:55:48.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34bb400700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f34b4013770 tx=0x7f34b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b28dbf1827065a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.252+0000 7f34a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4025030 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ac0ad5e0 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f34b400fd70 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f34b4032050 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34cc3f8b80 10 client.16506 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5629dd72c060 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.253+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ac0ad5e0 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f34b401f040 con 0x5629dd8137d0 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 10 client.16506 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 10 client.16506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:55:48.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 20 client.16506 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd826d40 0x5629dd8291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 4 client.16506 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629dd76dd90 con 0x5629dd826d40 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.254+0000 7f34baa00700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd826d40 0x5629dd8291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.255+0000 7f34baa00700 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd826d40 0x5629dd8291c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ec35793ba492286 server_cookie=18e71da876fc2767 in_seq=0 out_seq=0 2024-06-02T15:55:48.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.255+0000 7f34a8c00700 10 client.16506 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:48.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.255+0000 7f34a8c00700 1 -- 172.21.15.90:0/1021472685 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629dd76dd90 con 0x5629dd826d40 2024-06-02T15:55:48.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.255+0000 7f34a8c00700 10 client.16506 handle_command_reply: tid=0 2024-06-02T15:55:48.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 client.16506 shutdown 2024-06-02T15:55:48.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 msgr2=0x5629dd816d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f34b4013770 tx=0x7f34b4009260 comp rx=0 tx=0).stop 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 shutdown_connections 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd8137d0 0x5629dd816d70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd826d40 0x5629dd8291c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd819490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1021472685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd812eb0 0x5629dd816830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 >> 172.21.15.90:0/1021472685 conn(0x5629dd73bbe0 msgr2=0x5629dd80c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34a8c00700 0 client.16506 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 shutdown_connections 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.256+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1021472685 wait complete. 2024-06-02T15:55:48.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.257+0000 7f34cc3f8b80 20 client.16506 trim_cache size 0 max 16384 2024-06-02T15:55:48.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 Processor -- start 2024-06-02T15:55:48.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 -- start start 2024-06-02T15:55:48.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd80d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b3f80 con 0x5629dd812eb0 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b1750 con 0x5629dd80d880 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.273+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1af5a0 con 0x5629dd8137d0 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39482/0 (socket says 172.21.15.90:39482) 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 -- 172.21.15.90:0/3188635423 learned_addr learned my addr 172.21.15.90:0/3188635423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34baa00700 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd80d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34bb400700 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 -- 172.21.15.90:0/3188635423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 msgr2=0x5629dd82f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 -- 172.21.15.90:0/3188635423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 msgr2=0x5629dd80d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd80d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.274+0000 7f34b2000700 1 -- 172.21.15.90:0/3188635423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629dd8247f0 con 0x5629dd812eb0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.275+0000 7f34b2000700 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009990 tx=0x7f34a401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d09658f53babd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.275+0000 7f34ab400700 1 -- 172.21.15.90:0/3188635423 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4029020 con 0x5629dd812eb0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.275+0000 7f34ab400700 1 -- 172.21.15.90:0/3188635423 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a4003680 con 0x5629dd812eb0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.275+0000 7f34ab400700 1 -- 172.21.15.90:0/3188635423 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40256a0 con 0x5629dd812eb0 2024-06-02T15:55:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3188635423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 msgr2=0x5629dd82eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009990 tx=0x7f34a401a040 comp rx=0 tx=0).stop 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3188635423 shutdown_connections 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd80d880 0x5629dd80d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/3188635423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd812eb0 0x5629dd82eae0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3188635423 >> 172.21.15.90:0/3188635423 conn(0x5629dd73bbe0 msgr2=0x5629dd8109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3188635423 shutdown_connections 2024-06-02T15:55:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.276+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/3188635423 wait complete. 2024-06-02T15:55:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 -- start start 2024-06-02T15:55:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 Processor -- start 2024-06-02T15:55:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd82eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1afc10 con 0x5629dd80d880 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b1080 con 0x5629dd812eb0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.277+0000 7f34cc3f8b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b02e0 con 0x5629dd8137d0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34cc3f8b80 10 client.0 _ll_register_callbacks cb 0x5629dd705c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39488/0 (socket says 172.21.15.90:39488) 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 -- 172.21.15.90:0/1866536544 learned_addr learned my addr 172.21.15.90:0/1866536544 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34b2000700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd82eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34bb400700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 -- 172.21.15.90:0/1866536544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 msgr2=0x5629dd82f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 -- 172.21.15.90:0/1866536544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 msgr2=0x5629dd82eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd82eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34baa00700 1 -- 172.21.15.90:0/1866536544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629dd8909b0 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.278+0000 7f34bb400700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34baa00700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0dfa80 tx=0x7f34ac0a8e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dc1661591670d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0ad5e0 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f34ac0a9490 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a4003680 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f34ac0a9690 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34cc3f8b80 10 client.16512 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5629dd891c90 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.279+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a4003680 con 0x5629dd80d880 2024-06-02T15:55:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f34ac0ad120 con 0x5629dd80d880 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 10 client.16512 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 10 client.16512 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 20 client.16512 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd891fc0 0x5629dd8943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 4 client.16512 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629dd767570 con 0x5629dd891fc0 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.280+0000 7f34b2000700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd891fc0 0x5629dd8943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34b2000700 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd891fc0 0x5629dd8943d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5999cf4f9b8d3fa2 server_cookie=ca75c286c8986e8a in_seq=0 out_seq=0 2024-06-02T15:55:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34a8c00700 10 client.16512 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34a8c00700 1 -- 172.21.15.90:0/1866536544 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629dd767570 con 0x5629dd891fc0 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34a8c00700 10 client.16512 handle_command_reply: tid=0 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 client.16512 shutdown 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 msgr2=0x5629dd82e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0dfa80 tx=0x7f34ac0a8e90 comp rx=0 tx=0).stop 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 shutdown_connections 2024-06-02T15:55:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5629dd8137d0 0x5629dd82f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629dd80d880 0x5629dd82e620 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x5629dd891fc0 0x5629dd8943d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 --2- 172.21.15.90:0/1866536544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5629dd812eb0 0x5629dd82eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 >> 172.21.15.90:0/1866536544 conn(0x5629dd73bbe0 msgr2=0x5629dd813bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34a8c00700 0 client.16512 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.281+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 shutdown_connections 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34cc3f8b80 1 -- 172.21.15.90:0/1866536544 wait complete. 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34cc3f8b80 20 client.16512 trim_cache size 0 max 16384 2024-06-02T15:55:48.282 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 msgr2=0x7f34a00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 0x7f34a00b4dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f34b40097a0 tx=0x7f34b4009cc0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 msgr2=0x7f34bc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.282+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ef10 tx=0x7f34ac00c3a0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 shutdown_connections 2024-06-02T15:55:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f34a00b2900 0x7f34a00b4dc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f34bc1095d0 0x7f34bc103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34bc06bc10 0x7f34bc1028f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 --2- 172.21.15.90:0/2226342240 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34bc06c5d0 0x7f34bc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 >> 172.21.15.90:0/2226342240 conn(0x7f34bc0793f0 msgr2=0x7f34bc101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 shutdown_connections 2024-06-02T15:55:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:48.283+0000 7f34c1a00700 1 -- 172.21.15.90:0/2226342240 wait complete. 2024-06-02T15:55:49.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:55:49.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:49 smithi120 ceph-mon[36451]: pgmap v859: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:55:49.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:49 smithi135 ceph-mon[36839]: pgmap v859: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:55:49.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:49 smithi090 ceph-mon[31595]: pgmap v859: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:55:49.967 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:50.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:50 smithi090 ceph-mon[31595]: pgmap v860: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:50.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:50 smithi120 ceph-mon[36451]: pgmap v860: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:50.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:50 smithi135 ceph-mon[36839]: pgmap v860: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:55:52.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:52 smithi090 ceph-mon[31595]: pgmap v861: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:52.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:52 smithi120 ceph-mon[36451]: pgmap v861: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:52.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:52 smithi135 ceph-mon[36839]: pgmap v861: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:54.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:54 smithi090 ceph-mon[31595]: pgmap v862: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:54.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:54 smithi120 ceph-mon[36451]: pgmap v862: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:54.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:54 smithi135 ceph-mon[36839]: pgmap v862: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 -- 172.21.15.90:0/1079346007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc4076230 msgr2=0x7f8fc410f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 --2- 172.21.15.90:0/1079346007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc4076230 0x7f8fc410f7c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009990 tx=0x7f8fb0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 -- 172.21.15.90:0/1079346007 shutdown_connections 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 --2- 172.21.15.90:0/1079346007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc410fd00 0x7f8fc4112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 --2- 172.21.15.90:0/1079346007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc4075cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 --2- 172.21.15.90:0/1079346007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc4076230 0x7f8fc410f7c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 -- 172.21.15.90:0/1079346007 >> 172.21.15.90:0/1079346007 conn(0x7f8fc40fef90 msgr2=0x7f8fc41013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.592+0000 7f8fc9800700 1 -- 172.21.15.90:0/1079346007 shutdown_connections 2024-06-02T15:55:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.593+0000 7f8fc9800700 1 -- 172.21.15.90:0/1079346007 wait complete. 2024-06-02T15:55:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.594+0000 7f8fc9800700 1 Processor -- start 2024-06-02T15:55:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.594+0000 7f8fc9800700 1 -- start start 2024-06-02T15:55:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc4076230 0x7f8fc419c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc410fd00 0x7f8fc41a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41158a0 con 0x7f8fc410fd00 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4115720 con 0x7f8fc4075910 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc9800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4115a20 con 0x7f8fc4076230 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:48570/0 (socket says 172.21.15.90:48570) 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc3e00700 1 -- 172.21.15.90:0/3915086830 learned_addr learned my addr 172.21.15.90:0/3915086830 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc8e00700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc410fd00 0x7f8fc41a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.595+0000 7f8fc3400700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc4076230 0x7f8fc419c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 -- 172.21.15.90:0/3915086830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc4076230 msgr2=0x7f8fc419c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc4076230 0x7f8fc419c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 -- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc410fd00 msgr2=0x7f8fc41a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc410fd00 0x7f8fc41a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 -- 172.21.15.90:0/3915086830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41a0d30 con 0x7f8fc4075910 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400eb30 tx=0x7f8fb400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc0c00700 1 -- 172.21.15.90:0/3915086830 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400cbd0 con 0x7f8fc4075910 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc0c00700 1 -- 172.21.15.90:0/3915086830 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb40044d0 con 0x7f8fc4075910 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a1020 con 0x7f8fc4075910 2024-06-02T15:55:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc0c00700 1 -- 172.21.15.90:0/3915086830 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003e30 con 0x7f8fc4075910 2024-06-02T15:55:55.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.596+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a88a0 con 0x7f8fc4075910 2024-06-02T15:55:55.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.600+0000 7f8fc0c00700 1 -- 172.21.15.90:0/3915086830 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8fb4004000 con 0x7f8fc4075910 2024-06-02T15:55:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.602+0000 7f8fc0c00700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.602+0000 7f8fc0c00700 1 -- 172.21.15.90:0/3915086830 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8fb40e29b0 con 0x7f8fc4075910 2024-06-02T15:55:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.602+0000 7f8fc3400700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.602+0000 7f8fd5793b80 1 Processor -- start 2024-06-02T15:55:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 -- start start 2024-06-02T15:55:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fc3400700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 0x7f8fa80b4d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4111040 tx=0x7f8fb000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f4390 0x55e7166f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e716651e40 con 0x55e7166f4390 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e7166419e0 con 0x55e7166fa0e0 2024-06-02T15:55:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e716645a20 con 0x55e7166f97c0 2024-06-02T15:55:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:34756/0 (socket says 172.21.15.90:34756) 2024-06-02T15:55:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fc3e00700 1 -- 172.21.15.90:0/4052689329 learned_addr learned my addr 172.21.15.90:0/4052689329 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.603+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc8e00700 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f4390 0x55e7166f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 -- 172.21.15.90:0/4052689329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 msgr2=0x55e7166f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 -- 172.21.15.90:0/4052689329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f4390 msgr2=0x55e7166f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f4390 0x55e7166f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 -- 172.21.15.90:0/4052689329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71660a420 con 0x55e7166f97c0 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3400700 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.604+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009680 tx=0x7f8fb40a9990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca615ec053ce8ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.605+0000 7f8fb9600700 1 -- 172.21.15.90:0/4052689329 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a9330 con 0x55e7166f97c0 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.605+0000 7f8fb9600700 1 -- 172.21.15.90:0/4052689329 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb40b2370 con 0x55e7166f97c0 2024-06-02T15:55:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.605+0000 7f8fb9600700 1 -- 172.21.15.90:0/4052689329 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a5350 con 0x55e7166f97c0 2024-06-02T15:55:55.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.605+0000 7f8fd5793b80 1 -- 172.21.15.90:0/4052689329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 msgr2=0x55e7166f9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.605+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009680 tx=0x7f8fb40a9990 comp rx=0 tx=0).stop 2024-06-02T15:55:55.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 -- 172.21.15.90:0/4052689329 shutdown_connections 2024-06-02T15:55:55.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f4390 0x55e7166f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f97c0 0x55e7166f9ba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/4052689329 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 -- 172.21.15.90:0/4052689329 >> 172.21.15.90:0/4052689329 conn(0x55e716621be0 msgr2=0x55e7166ee810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 -- 172.21.15.90:0/4052689329 shutdown_connections 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 -- 172.21.15.90:0/4052689329 wait complete. 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 -- start start 2024-06-02T15:55:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.606+0000 7f8fd5793b80 1 Processor -- start 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e7166fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e716547fa0 con 0x55e7166f97c0 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e7165e2760 con 0x55e7166fa0e0 2024-06-02T15:55:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e716700c00 con 0x55e7166f4390 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.607+0000 7f8fd5793b80 10 client.0 _ll_register_callbacks cb 0x55e7165ebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e7166fd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:34762/0 (socket says 172.21.15.90:34762) 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 -- 172.21.15.90:0/2374543248 learned_addr learned my addr 172.21.15.90:0/2374543248 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc8e00700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 -- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 msgr2=0x55e7166fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 -- 172.21.15.90:0/2374543248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 msgr2=0x55e7166fd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e7166fd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 -- 172.21.15.90:0/2374543248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7165918d0 con 0x55e7166f4390 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40038d0 tx=0x7f8fb40aa0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5df281d0436f03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.608+0000 7f8fae000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:55:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb406d020 con 0x55e7166f4390 2024-06-02T15:55:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8fb40b27a0 con 0x55e7166f4390 2024-06-02T15:55:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb40b2370 con 0x55e7166f4390 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8fb40a5350 con 0x55e7166f4390 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fd5793b80 10 client.26758 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e716612060 con 0x55e7166f4390 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb40b2370 con 0x55e7166f4390 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.609+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8fb40b2050 con 0x55e7166f4390 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 10 client.26758 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 10 client.26758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 20 client.26758 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e71670d5a0 0x55e71670f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 4 client.26758 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e716653d90 con 0x55e71670d5a0 2024-06-02T15:55:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.610+0000 7f8fc3400700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e71670d5a0 0x55e71670f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.611+0000 7f8fc3400700 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e71670d5a0 0x55e71670f990 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8edf3bc98bac05 server_cookie=2495e340469e1953 in_seq=0 out_seq=0 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.611+0000 7f8fae000700 10 client.26758 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.611+0000 7f8fae000700 1 -- 172.21.15.90:0/2374543248 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e716653d90 con 0x55e71670d5a0 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.611+0000 7f8fae000700 10 client.26758 handle_command_reply: tid=0 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 client.26758 shutdown 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 msgr2=0x55e7166ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40038d0 tx=0x7f8fb40aa0d0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 shutdown_connections 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e71670d5a0 0x55e71670f990 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166fa0e0 0x55e7166fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166ffe00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2374543248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e7166fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fae000700 0 client.26758 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 >> 172.21.15.90:0/2374543248 conn(0x55e716621be0 msgr2=0x55e7166edfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 shutdown_connections 2024-06-02T15:55:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2374543248 wait complete. 2024-06-02T15:55:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.612+0000 7f8fd5793b80 20 client.26758 trim_cache size 0 max 16384 2024-06-02T15:55:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 Processor -- start 2024-06-02T15:55:55.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 -- start start 2024-06-02T15:55:55.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 0x55e7166ee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41b3de0 con 0x55e7166fa0e0 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41b15b0 con 0x55e7166f97c0 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.629+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41af400 con 0x55e7166f4390 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 0x55e7166ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:48604/0 (socket says 172.21.15.90:48604) 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 -- 172.21.15.90:0/2201185866 learned_addr learned my addr 172.21.15.90:0/2201185866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 0x55e7166ee740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41384/0 (socket says 172.21.15.90:41384) 2024-06-02T15:55:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 -- 172.21.15.90:0/2201185866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 msgr2=0x55e7166f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 -- 172.21.15.90:0/2201185866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 msgr2=0x55e7166ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 0x55e7166ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.630+0000 7f8fc3400700 1 -- 172.21.15.90:0/2201185866 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7166ef2e0 con 0x55e7166f97c0 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.631+0000 7f8fc3400700 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009750 tx=0x7f8fb0022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711cb9de656c5216 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.631+0000 7f8fb8c00700 1 -- 172.21.15.90:0/2201185866 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0003680 con 0x55e7166f97c0 2024-06-02T15:55:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.631+0000 7f8fb8c00700 1 -- 172.21.15.90:0/2201185866 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb0003cd0 con 0x55e7166f97c0 2024-06-02T15:55:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.631+0000 7f8fb8c00700 1 -- 172.21.15.90:0/2201185866 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f460 con 0x55e7166f97c0 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2201185866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 msgr2=0x55e7166fabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009750 tx=0x7f8fb0022000 comp rx=0 tx=0).stop 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2201185866 shutdown_connections 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166fa0e0 0x55e7166ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166f4390 0x55e7166f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/2201185866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f97c0 0x55e7166fabd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2201185866 >> 172.21.15.90:0/2201185866 conn(0x55e716621be0 msgr2=0x55e7166fa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2201185866 shutdown_connections 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.632+0000 7f8fd5793b80 1 -- 172.21.15.90:0/2201185866 wait complete. 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 -- start start 2024-06-02T15:55:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 Processor -- start 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 0x55e716712f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 0x55e716710850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41afa70 con 0x55e7166f97c0 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41b0ee0 con 0x55e7166f4390 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.633+0000 7f8fd5793b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41b0140 con 0x55e7166fa0e0 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fd5793b80 10 client.0 _ll_register_callbacks cb 0x55e7165ebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:55:55.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41390/0 (socket says 172.21.15.90:41390) 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 -- 172.21.15.90:0/1423631084 learned_addr learned my addr 172.21.15.90:0/1423631084 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc8e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 0x55e716710850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 0x55e716712f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 -- 172.21.15.90:0/1423631084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 msgr2=0x55e716710850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 0x55e716710850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 -- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 msgr2=0x55e716712f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 0x55e716712f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3400700 1 -- 172.21.15.90:0/1423631084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e716710f50 con 0x55e7166f97c0 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc8e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 0x55e716710850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.634+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 0x55e716712f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fc3400700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005510 tx=0x7f8fb00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26180959cfbc2abe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:55:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fae000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0003a90 con 0x55e7166f97c0 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8fb0034450 con 0x55e7166f97c0 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb0003cd0 con 0x55e7166f97c0 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8fb004b050 con 0x55e7166f97c0 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.635+0000 7f8fd5793b80 10 client.16530 fetch_fsmap learned FSMap version 41 2024-06-02T15:55:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e716777cf0 con 0x55e7166f97c0 2024-06-02T15:55:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb0003cd0 con 0x55e7166f97c0 2024-06-02T15:55:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f8fb000f9d0 con 0x55e7166f97c0 2024-06-02T15:55:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:55:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 20 client.16530 populate_metadata read hostname 'smithi090' 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e716778420 0x55e71677a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 4 client.16530 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.636+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e71664d570 con 0x55e716778420 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e716778420 0x55e71677a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fae000700 10 client.16530 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fc3e00700 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e716778420 0x55e71677a840 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3deaf44c20ee866d server_cookie=fa836a26295235bb in_seq=0 out_seq=0 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fae000700 1 -- 172.21.15.90:0/1423631084 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e71664d570 con 0x55e716778420 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fae000700 10 client.16530 handle_command_reply: tid=0 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fd5793b80 1 client.16530 shutdown 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.637+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 msgr2=0x55e716710310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005510 tx=0x7f8fb00057e0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 shutdown_connections 2024-06-02T15:55:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x55e716778420 0x55e71677a840 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7166fa0e0 0x55e716710850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e7166f4390 0x55e716712f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 --2- 172.21.15.90:0/1423631084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e7166f97c0 0x55e716710310 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 >> 172.21.15.90:0/1423631084 conn(0x55e716621be0 msgr2=0x55e7166fad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fae000700 0 client.16530 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 shutdown_connections 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 1 -- 172.21.15.90:0/1423631084 wait complete. 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.638+0000 7f8fd5793b80 20 client.16530 trim_cache size 0 max 16384 2024-06-02T15:55:55.638 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 msgr2=0x7f8fa80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 0x7f8fa80b4d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4111040 tx=0x7f8fb000b260 comp rx=0 tx=0).stop 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 msgr2=0x7f8fc419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400eb30 tx=0x7f8fb400ee40 comp rx=0 tx=0).stop 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 shutdown_connections 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fc410fd00 0x7f8fc41a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8fc4075910 0x7f8fc419bd70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 --2- 172.21.15.90:0/3915086830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fc4076230 0x7f8fc419c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:55:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 >> 172.21.15.90:0/3915086830 conn(0x7f8fc40fef90 msgr2=0x7f8fc40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:55:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 shutdown_connections 2024-06-02T15:55:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:55:55.639+0000 7f8fc9800700 1 -- 172.21.15.90:0/3915086830 wait complete. 2024-06-02T15:55:56.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:56 smithi090 ceph-mon[31595]: pgmap v863: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:56.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:56 smithi120 ceph-mon[36451]: pgmap v863: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:56.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:56 smithi135 ceph-mon[36839]: pgmap v863: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:55:57.833 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:55:57.833 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:55:58.285 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:55:58.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:55:58 smithi120 ceph-mon[36451]: pgmap v864: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:55:58.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:55:58 smithi090 ceph-mon[31595]: pgmap v864: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:55:58.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:55:58 smithi135 ceph-mon[36839]: pgmap v864: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:00.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:00 smithi090 ceph-mon[31595]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:00.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:00 smithi120 ceph-mon[36451]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:00.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:00 smithi135 ceph-mon[36839]: pgmap v865: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:02.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:02 smithi090 ceph-mon[31595]: pgmap v866: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 -- 172.21.15.90:0/1554976009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 msgr2=0x7f3ab8105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 --2- 172.21.15.90:0/1554976009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab8105be0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009990 tx=0x7f3aa4009ca0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 -- 172.21.15.90:0/1554976009 shutdown_connections 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 --2- 172.21.15.90:0/1554976009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab8106120 0x7f3ab8112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 --2- 172.21.15.90:0/1554976009 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab810b760 0x7f3ab810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 --2- 172.21.15.90:0/1554976009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab8105be0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 -- 172.21.15.90:0/1554976009 >> 172.21.15.90:0/1554976009 conn(0x7f3ab8101250 msgr2=0x7f3ab8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 -- 172.21.15.90:0/1554976009 shutdown_connections 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.721+0000 7f3abfe00700 1 -- 172.21.15.90:0/1554976009 wait complete. 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.722+0000 7f3abfe00700 1 Processor -- start 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.722+0000 7f3abfe00700 1 -- start start 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 0x7f3ab819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab810b760 0x7f3ab81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8115990 con 0x7f3ab8105760 2024-06-02T15:56:02.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8115810 con 0x7f3ab8106120 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abfe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8115b10 con 0x7f3ab810b760 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54726/0 (socket says 172.21.15.90:54726) 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 -- 172.21.15.90:0/3715014998 learned_addr learned my addr 172.21.15.90:0/3715014998 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abde00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab810b760 0x7f3ab81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abca00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 0x7f3ab819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 -- 172.21.15.90:0/3715014998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab810b760 msgr2=0x7f3ab81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab810b760 0x7f3ab81a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 -- 172.21.15.90:0/3715014998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 msgr2=0x7f3ab819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 0x7f3ab819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abd400700 1 -- 172.21.15.90:0/3715014998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a0e70 con 0x7f3ab8105760 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.723+0000 7f3abca00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 0x7f3ab819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:02.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3abd400700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d980 tx=0x7f3aac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3ab6000700 1 -- 172.21.15.90:0/3715014998 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac016070 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3ab6000700 1 -- 172.21.15.90:0/3715014998 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac004830 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a1160 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3ab6000700 1 -- 172.21.15.90:0/3715014998 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac005540 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.724+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a89e0 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.728+0000 7f3ab6000700 1 -- 172.21.15.90:0/3715014998 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3aac004220 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.729+0000 7f3ab6000700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 0x7f3aa00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.729+0000 7f3aca69fb80 1 Processor -- start 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 -- start start 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3ab6000700 1 -- 172.21.15.90:0/3715014998 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3aac0e2350 con 0x7f3ab8105760 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3abca00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 0x7f3aa00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 0x5632e38f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 0x5632e38fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5632e3857ec0 con 0x5632e38ff690 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5632e3847a60 con 0x5632e38fa1c0 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632e384baa0 con 0x5632e38fffb0 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54734/0 (socket says 172.21.15.90:54734) 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.730+0000 7f3abd400700 1 -- 172.21.15.90:0/3238567194 learned_addr learned my addr 172.21.15.90:0/3238567194 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abde00700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 0x5632e38fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abca00700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 0x5632e38f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 -- 172.21.15.90:0/3238567194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 msgr2=0x5632e38f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 0x5632e38f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 -- 172.21.15.90:0/3238567194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 msgr2=0x5632e38fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 0x5632e38fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 -- 172.21.15.90:0/3238567194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632e38104a0 con 0x5632e38ff690 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abde00700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 0x5632e38fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abca00700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 0x5632e38f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abca00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 0x7f3aa00b4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8103860 tx=0x7f3aa400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3abd400700 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d950 tx=0x7f3aac0a90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9502ddad8a7c1b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3aaaa00700 1 -- 172.21.15.90:0/3238567194 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0e2150 con 0x5632e38ff690 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.731+0000 7f3aaaa00700 1 -- 172.21.15.90:0/3238567194 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac0ad910 con 0x5632e38ff690 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.732+0000 7f3aaaa00700 1 -- 172.21.15.90:0/3238567194 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0a9820 con 0x5632e38ff690 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.732+0000 7f3aca69fb80 1 -- 172.21.15.90:0/3238567194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 msgr2=0x5632e38ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.732+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d950 tx=0x7f3aac0a90f0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 -- 172.21.15.90:0/3238567194 shutdown_connections 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fa1c0 0x5632e38fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e38ffa70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/3238567194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fffb0 0x5632e38f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 -- 172.21.15.90:0/3238567194 >> 172.21.15.90:0/3238567194 conn(0x5632e3827c60 msgr2=0x5632e38f4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 -- 172.21.15.90:0/3238567194 shutdown_connections 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 -- 172.21.15.90:0/3238567194 wait complete. 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 -- start start 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.733+0000 7f3aca69fb80 1 Processor -- start 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38ff690 0x5632e39030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5632e374e020 con 0x5632e38fa1c0 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5632e37e87e0 con 0x5632e38fffb0 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632e3906a00 con 0x5632e38ff690 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3aca69fb80 10 client.0 _ll_register_callbacks cb 0x5632e37f1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3abd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54740/0 (socket says 172.21.15.90:54740) 2024-06-02T15:56:02.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3abd400700 1 -- 172.21.15.90:0/11533743 learned_addr learned my addr 172.21.15.90:0/11533743 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3abca00700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38ff690 0x5632e39030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.734+0000 7f3abde00700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 -- 172.21.15.90:0/11533743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38ff690 msgr2=0x5632e39030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38ff690 0x5632e39030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 -- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 msgr2=0x5632e39035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 -- 172.21.15.90:0/11533743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632e3797950 con 0x5632e38fa1c0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3abd400700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0a55e0 tx=0x7f3aac0e21d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbfff21c50e83f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3a9f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0c2070 con 0x5632e38fa1c0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3aac0a99d0 con 0x5632e38fa1c0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.735+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac0ad910 con 0x5632e38fa1c0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3aac0910b0 con 0x5632e38fa1c0 2024-06-02T15:56:02.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 10 client.16542 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5632e38180e0 con 0x5632e38fa1c0 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac0ad910 con 0x5632e38fa1c0 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3aac0ca360 con 0x5632e38fa1c0 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 10 client.16542 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 20 client.16542 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e3913380 0x5632e3915770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 4 client.16542 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.736+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632e3859e10 con 0x5632e3913380 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.737+0000 7f3abca00700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e3913380 0x5632e3915770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.737+0000 7f3abca00700 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e3913380 0x5632e3915770 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1474ca772d5dc208 server_cookie=aa6b06c7876ac82a in_seq=0 out_seq=0 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.737+0000 7f3a9f400700 10 client.16542 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3a9f400700 1 -- 172.21.15.90:0/11533743 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5632e3859e10 con 0x5632e3913380 2024-06-02T15:56:02.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3a9f400700 10 client.16542 handle_command_reply: tid=0 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 client.16542 shutdown 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 msgr2=0x5632e3905d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0a55e0 tx=0x7f3aac0e21d0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 shutdown_connections 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e3913380 0x5632e3915770 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3a9f400700 0 client.16542 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fa1c0 0x5632e3905d00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/11533743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38ff690 0x5632e39030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.738+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 >> 172.21.15.90:0/11533743 conn(0x5632e3827c60 msgr2=0x5632e38f3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.739+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 shutdown_connections 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.739+0000 7f3aca69fb80 1 -- 172.21.15.90:0/11533743 wait complete. 2024-06-02T15:56:02.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.739+0000 7f3aca69fb80 20 client.16542 trim_cache size 0 max 16384 2024-06-02T15:56:02.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 Processor -- start 2024-06-02T15:56:02.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 -- start start 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38ff690 0x5632e3902220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 0x5632e38f4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b4000 con 0x5632e38fffb0 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b17d0 con 0x5632e38ff690 2024-06-02T15:56:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.762+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81af620 con 0x5632e38fa1c0 2024-06-02T15:56:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 0x5632e38f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:60792/0 (socket says 172.21.15.90:60792) 2024-06-02T15:56:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 0x5632e38f4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54756/0 (socket says 172.21.15.90:54756) 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abca00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38ff690 0x5632e3902220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 -- 172.21.15.90:0/84989864 learned_addr learned my addr 172.21.15.90:0/84989864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 -- 172.21.15.90:0/84989864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38ff690 msgr2=0x5632e3902220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38ff690 0x5632e3902220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 -- 172.21.15.90:0/84989864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 msgr2=0x5632e38f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 0x5632e38f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 -- 172.21.15.90:0/84989864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632e38f5520 con 0x5632e38fa1c0 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.763+0000 7f3abd400700 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0a67d0 tx=0x7f3aac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b425302c9a697856 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.764+0000 7f3aaa000700 1 -- 172.21.15.90:0/84989864 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0c2040 con 0x5632e38fa1c0 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.764+0000 7f3aaa000700 1 -- 172.21.15.90:0/84989864 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac08fe50 con 0x5632e38fa1c0 2024-06-02T15:56:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.764+0000 7f3aaa000700 1 -- 172.21.15.90:0/84989864 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0a4ed0 con 0x5632e38fa1c0 2024-06-02T15:56:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- 172.21.15.90:0/84989864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 msgr2=0x5632e38f9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0a67d0 tx=0x7f3aac004750 comp rx=0 tx=0).stop 2024-06-02T15:56:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- 172.21.15.90:0/84989864 shutdown_connections 2024-06-02T15:56:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38fffb0 0x5632e38f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e38f9ba0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/84989864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38ff690 0x5632e3902220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- 172.21.15.90:0/84989864 >> 172.21.15.90:0/84989864 conn(0x5632e3827c60 msgr2=0x5632e3900a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- 172.21.15.90:0/84989864 shutdown_connections 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- 172.21.15.90:0/84989864 wait complete. 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.765+0000 7f3aca69fb80 1 -- start start 2024-06-02T15:56:02.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 Processor -- start 2024-06-02T15:56:02.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e390c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81afc90 con 0x5632e38ff690 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b1100 con 0x5632e38fffb0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0360 con 0x5632e38fa1c0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.766+0000 7f3aca69fb80 10 client.0 _ll_register_callbacks cb 0x5632e37f1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54772/0 (socket says 172.21.15.90:54772) 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abca00700 1 -- 172.21.15.90:0/832106515 learned_addr learned my addr 172.21.15.90:0/832106515 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abd400700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e390c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 -- 172.21.15.90:0/832106515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 msgr2=0x5632e390c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e390c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 -- 172.21.15.90:0/832106515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 msgr2=0x5632e3908c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 -- 172.21.15.90:0/832106515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632e3909810 con 0x5632e38fffb0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abca00700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abd400700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e390c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3abde00700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39091d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0007c10 tx=0x7f3ab0007c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c56388c4fdda36d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.767+0000 7f3a9f400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0004bb0 con 0x5632e38fffb0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac08fe50 con 0x5632e38fffb0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3ab0004d50 con 0x5632e38fffb0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ab00175b0 con 0x5632e38fffb0 2024-06-02T15:56:02.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3aca69fb80 10 client.26846 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:02.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5632e397ebb0 con 0x5632e38fffb0 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.768+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac08fe50 con 0x5632e38fffb0 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3ab00219b0 con 0x5632e38fffb0 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 10 client.26846 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 10 client.26846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 20 client.26846 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:02.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e397f330 0x5632e3981750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 4 client.26846 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632e38535f0 con 0x5632e397f330 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3abd400700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e397f330 0x5632e3981750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3abd400700 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e397f330 0x5632e3981750 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eabbb147e605502 server_cookie=c986ccb73bbc311a in_seq=0 out_seq=0 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.769+0000 7f3a9f400700 10 client.26846 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.770+0000 7f3a9f400700 1 -- 172.21.15.90:0/832106515 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632e38535f0 con 0x5632e397f330 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.770+0000 7f3a9f400700 10 client.26846 handle_command_reply: tid=0 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.770+0000 7f3aca69fb80 1 client.26846 shutdown 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 msgr2=0x5632e39091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39091d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0007c10 tx=0x7f3ab0007c40 comp rx=0 tx=0).stop 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 shutdown_connections 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5632e397f330 0x5632e3981750 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5632e38fffb0 0x5632e39091d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632e38fa1c0 0x5632e390c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 --2- 172.21.15.90:0/832106515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5632e38ff690 0x5632e3908c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3a9f400700 0 client.26846 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 >> 172.21.15.90:0/832106515 conn(0x5632e3827c60 msgr2=0x5632e39046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 shutdown_connections 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 1 -- 172.21.15.90:0/832106515 wait complete. 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.771+0000 7f3aca69fb80 20 client.26846 trim_cache size 0 max 16384 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:56:02.771 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "a82eaf23-329c-474b-a41b-421640f5fe4a", 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 msgr2=0x7f3aa00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 0x7f3aa00b4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8103860 tx=0x7f3aa400b540 comp rx=0 tx=0).stop 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 msgr2=0x7f3ab819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00d980 tx=0x7f3aac00dc90 comp rx=0 tx=0).stop 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 shutdown_connections 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3aa00b2880 0x7f3aa00b4d40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ab810b760 0x7f3ab81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ab8105760 0x7f3ab819be60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 --2- 172.21.15.90:0/3715014998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3ab8106120 0x7f3ab819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:02.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.772+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 >> 172.21.15.90:0/3715014998 conn(0x7f3ab8101250 msgr2=0x7f3ab8101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:02.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.773+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 shutdown_connections 2024-06-02T15:56:02.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:02.773+0000 7f3abfe00700 1 -- 172.21.15.90:0/3715014998 wait complete. 2024-06-02T15:56:02.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:02 smithi120 ceph-mon[36451]: pgmap v866: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:02 smithi135 ceph-mon[36839]: pgmap v866: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:03 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:56:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:03 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:03 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:56:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:03 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:03 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:56:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:03 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:03 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:56:03.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:03 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:03 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:56:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:03 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:03 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:56:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:03 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:04.286 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:56:04.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:04 smithi090 ceph-mon[31595]: pgmap v867: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s 2024-06-02T15:56:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:04.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:04 smithi120 ceph-mon[36451]: pgmap v867: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s 2024-06-02T15:56:04.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:04 smithi135 ceph-mon[36839]: pgmap v867: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s 2024-06-02T15:56:06.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:06 smithi090 ceph-mon[31595]: pgmap v868: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:56:06.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:06 smithi120 ceph-mon[36451]: pgmap v868: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:56:06.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:06 smithi135 ceph-mon[36839]: pgmap v868: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 10 op/s 2024-06-02T15:56:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:07 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:56:07.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:07 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:56:07.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:07 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:56:08.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:08 smithi090 ceph-mon[31595]: pgmap v869: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:08.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:08 smithi120 ceph-mon[36451]: pgmap v869: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:08.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:08 smithi135 ceph-mon[36839]: pgmap v869: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 -- 172.21.15.90:0/3640157770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4530109dd0 msgr2=0x7f453010a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 --2- 172.21.15.90:0/3640157770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4530109dd0 0x7f453010a1b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f45200099c0 tx=0x7f4520009cd0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 -- 172.21.15.90:0/3640157770 shutdown_connections 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 --2- 172.21.15.90:0/3640157770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 0x7f4530116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 --2- 172.21.15.90:0/3640157770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4530109dd0 0x7f453010a1b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 --2- 172.21.15.90:0/3640157770 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f453010a780 0x7f453010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 -- 172.21.15.90:0/3640157770 >> 172.21.15.90:0/3640157770 conn(0x7f4530076d10 msgr2=0x7f4530108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 -- 172.21.15.90:0/3640157770 shutdown_connections 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.864+0000 7f4536800700 1 -- 172.21.15.90:0/3640157770 wait complete. 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.865+0000 7f4536800700 1 Processor -- start 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 -- start start 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 0x7f45301a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 0x7f45301a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4530119d90 con 0x7f453010a780 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4530119c10 con 0x7f4530109dd0 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f4536800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4530119f10 con 0x7f453010eeb0 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f452ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f452ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54786/0 (socket says 172.21.15.90:54786) 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f452ea00700 1 -- 172.21.15.90:0/2952334099 learned_addr learned my addr 172.21.15.90:0/2952334099 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f452f400700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 0x7f45301a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.866+0000 7f452fe00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 0x7f45301a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 -- 172.21.15.90:0/2952334099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 msgr2=0x7f45301a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 0x7f45301a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 -- 172.21.15.90:0/2952334099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 msgr2=0x7f45301a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 0x7f45301a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 -- 172.21.15.90:0/2952334099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45301a5290 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452f400700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 0x7f45301a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f452ea00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f451c00ea30 tx=0x7f451c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f4527e00700 1 -- 172.21.15.90:0/2952334099 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c00cbd0 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45301a5580 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f4527e00700 1 -- 172.21.15.90:0/2952334099 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0044d0 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f4527e00700 1 -- 172.21.15.90:0/2952334099 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c003dc0 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.867+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301ace00 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.872+0000 7f4527e00700 1 -- 172.21.15.90:0/2952334099 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f451c004000 con 0x7f453010a780 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.872+0000 7f45410adb80 1 Processor -- start 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.872+0000 7f45410adb80 1 -- start start 2024-06-02T15:56:08.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.872+0000 7f45410adb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db821e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f45410adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db8218260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f45410adb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f45410adb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559db8176e40 con 0x559db8218880 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f45410adb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559db81669e0 con 0x559db821deb0 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f45410adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559db816aa20 con 0x559db821e7d0 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db8218260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54788/0 (socket says 172.21.15.90:54788) 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 -- 172.21.15.90:0/3461621109 learned_addr learned my addr 172.21.15.90:0/3461621109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452fe00700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db821e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 -- 172.21.15.90:0/3461621109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 msgr2=0x559db8218260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db8218260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 -- 172.21.15.90:0/3461621109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 msgr2=0x559db821e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db821e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452ea00700 1 -- 172.21.15.90:0/3461621109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db812f420 con 0x559db8218880 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452f400700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db8218260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.873+0000 7f452fe00700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db821e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f4527e00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 0x7f45140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f452ea00700 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f451c0e8a90 tx=0x7f451c0e8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2146264c8419896 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f4527e00700 1 -- 172.21.15.90:0/2952334099 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f451c0e2740 con 0x7f453010a780 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f452fe00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 0x7f45140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f4524c00700 1 -- 172.21.15.90:0/3461621109 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a5810 con 0x559db8218880 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f4524c00700 1 -- 172.21.15.90:0/3461621109 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c00ce00 con 0x559db8218880 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.874+0000 7f4524c00700 1 -- 172.21.15.90:0/3461621109 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a90c0 con 0x559db8218880 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 -- 172.21.15.90:0/3461621109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 msgr2=0x559db8218ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f451c0e8a90 tx=0x7f451c0e8ac0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f452fe00700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 0x7f45140b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f45280075c0 tx=0x7f452800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 -- 172.21.15.90:0/3461621109 shutdown_connections 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218ce0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db821e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 --2- 172.21.15.90:0/3461621109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db8218260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 -- 172.21.15.90:0/3461621109 >> 172.21.15.90:0/3461621109 conn(0x559db8146be0 msgr2=0x559db8213010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 -- 172.21.15.90:0/3461621109 shutdown_connections 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.875+0000 7f45410adb80 1 -- 172.21.15.90:0/3461621109 wait complete. 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.876+0000 7f45410adb80 1 -- start start 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.876+0000 7f45410adb80 1 Processor -- start 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8224570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db8221e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559db806cfa0 con 0x559db821e7d0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x559db8107760 con 0x559db821deb0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559db8225240 con 0x559db8218880 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.877+0000 7f45410adb80 10 client.0 _ll_register_callbacks cb 0x559db8110c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db8221e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:57114/0 (socket says 172.21.15.90:57114) 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 -- 172.21.15.90:0/2252696177 learned_addr learned my addr 172.21.15.90:0/2252696177 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452f400700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8224570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 -- 172.21.15.90:0/2252696177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 msgr2=0x559db8224570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8224570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 -- 172.21.15.90:0/2252696177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 msgr2=0x559db8221e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db8221e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 -- 172.21.15.90:0/2252696177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db80b68d0 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f452ea00700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f451c0e5c20 tx=0x7f451c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb9b69fb42726e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.878+0000 7f4519600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a92a0 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f451c0cc110 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559db8178d90 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f451c0a9720 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 10 client.26858 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559db8137060 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559db8178d90 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f451c0e1040 con 0x559db821deb0 2024-06-02T15:56:08.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 10 client.26858 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 10 client.26858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 20 client.26858 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db8231c90 0x559db8234080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 4 client.26858 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.879+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db8172570 con 0x559db8231c90 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.880+0000 7f452f400700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db8231c90 0x559db8234080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.880+0000 7f4519600700 10 client.26858 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.880+0000 7f452f400700 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db8231c90 0x559db8234080 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83dcde8da3bd1034 server_cookie=177c23c4d0307fb0 in_seq=0 out_seq=0 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.881+0000 7f4519600700 1 -- 172.21.15.90:0/2252696177 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559db8172570 con 0x559db8231c90 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.881+0000 7f4519600700 10 client.26858 handle_command_reply: tid=0 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.881+0000 7f45410adb80 1 client.26858 shutdown 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 msgr2=0x559db8221910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f451c0e5c20 tx=0x7f451c00c6e0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 shutdown_connections 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db8231c90 0x559db8234080 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db8221e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8224570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f4519600700 0 client.26858 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 --2- 172.21.15.90:0/2252696177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8221910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 >> 172.21.15.90:0/2252696177 conn(0x559db8146be0 msgr2=0x559db8217250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 shutdown_connections 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 1 -- 172.21.15.90:0/2252696177 wait complete. 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.882+0000 7f45410adb80 20 client.26858 trim_cache size 0 max 16384 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.904+0000 7f45410adb80 1 Processor -- start 2024-06-02T15:56:08.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 -- start start 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db82130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db821ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b8440 con 0x559db8218880 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b5c10 con 0x559db821deb0 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f45410adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b3a60 con 0x559db821e7d0 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f452f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f452f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54800/0 (socket says 172.21.15.90:54800) 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f452f400700 1 -- 172.21.15.90:0/2401732736 learned_addr learned my addr 172.21.15.90:0/2401732736 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f452ea00700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db82130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.905+0000 7f452fe00700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db821ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 -- 172.21.15.90:0/2401732736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 msgr2=0x559db821ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db821ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 -- 172.21.15.90:0/2401732736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 msgr2=0x559db82130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db82130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 -- 172.21.15.90:0/2401732736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db822ec80 con 0x559db8218880 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452ea00700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db82130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f452f400700 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4520003760 tx=0x7f452001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145d944fc7d93 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f451be00700 1 -- 172.21.15.90:0/2401732736 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200052b0 con 0x559db8218880 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f451be00700 1 -- 172.21.15.90:0/2401732736 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45200058e0 con 0x559db8218880 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.906+0000 7f451be00700 1 -- 172.21.15.90:0/2401732736 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452000f6c0 con 0x559db8218880 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 -- 172.21.15.90:0/2401732736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 msgr2=0x559db8218260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4520003760 tx=0x7f452001a040 comp rx=0 tx=0).stop 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 -- 172.21.15.90:0/2401732736 shutdown_connections 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db821e7d0 0x559db821ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db8218880 0x559db8218260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 --2- 172.21.15.90:0/2401732736 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db82130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 -- 172.21.15.90:0/2401732736 >> 172.21.15.90:0/2401732736 conn(0x559db8146be0 msgr2=0x559db82129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 -- 172.21.15.90:0/2401732736 shutdown_connections 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.907+0000 7f45410adb80 1 -- 172.21.15.90:0/2401732736 wait complete. 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.908+0000 7f45410adb80 1 -- start start 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.908+0000 7f45410adb80 1 Processor -- start 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8235660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8235ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b40d0 con 0x559db821e7d0 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b5540 con 0x559db821deb0 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f45301b47a0 con 0x559db8218880 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f45410adb80 10 client.0 _ll_register_callbacks cb 0x559db8110c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452f400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8235660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54808/0 (socket says 172.21.15.90:54808) 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 -- 172.21.15.90:0/1121848516 learned_addr learned my addr 172.21.15.90:0/1121848516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452ea00700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8235ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 -- 172.21.15.90:0/1121848516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 msgr2=0x559db8235660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8235660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 -- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 msgr2=0x559db8235ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8235ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.909+0000 7f452fe00700 1 -- 172.21.15.90:0/1121848516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db8295ae0 con 0x559db821e7d0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f452fe00700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4528004690 tx=0x7f4528004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e9d3a724e954ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f4519600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:56:08.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528025cb0 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4528016400 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45200058e0 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4528036050 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f45410adb80 10 client.16566 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.910+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559db829cee0 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45200058e0 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f452802e510 con 0x559db821e7d0 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 10 client.16566 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 20 client.16566 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db829d640 0x559db829fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 4 client.16566 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559db7fe9250 con 0x559db829d640 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.911+0000 7f452f400700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db829d640 0x559db829fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f452f400700 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db829d640 0x559db829fa60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ec30ded54d304 server_cookie=5eceafc72bb6a38 in_seq=0 out_seq=0 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f4519600700 10 client.16566 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f4519600700 1 -- 172.21.15.90:0/1121848516 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559db7fe9250 con 0x559db829d640 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f4519600700 10 client.16566 handle_command_reply: tid=0 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 client.16566 shutdown 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 msgr2=0x559db82954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4528004690 tx=0x7f4528004b40 comp rx=0 tx=0).stop 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 shutdown_connections 2024-06-02T15:56:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x559db829d640 0x559db829fa60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559db821e7d0 0x559db82954a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559db8218880 0x559db8235660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f4519600700 0 client.16566 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 --2- 172.21.15.90:0/1121848516 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x559db821deb0 0x559db8235ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.912+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 >> 172.21.15.90:0/1121848516 conn(0x559db8146be0 msgr2=0x559db821ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 shutdown_connections 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f45410adb80 1 -- 172.21.15.90:0/1121848516 wait complete. 2024-06-02T15:56:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f45410adb80 20 client.16566 trim_cache size 0 max 16384 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 msgr2=0x7f45140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 0x7f45140b4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f45280075c0 tx=0x7f452800b040 comp rx=0 tx=0).stop 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 msgr2=0x7f45301a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.913+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f451c00ea30 tx=0x7f451c00ed40 comp rx=0 tx=0).stop 2024-06-02T15:56:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 shutdown_connections 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f45140b2880 0x7f45140b4d40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f453010eeb0 0x7f45301a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4530109dd0 0x7f45301a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 --2- 172.21.15.90:0/2952334099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f453010a780 0x7f45301a07c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 >> 172.21.15.90:0/2952334099 conn(0x7f4530076d10 msgr2=0x7f4530078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 shutdown_connections 2024-06-02T15:56:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:08.914+0000 7f4536800700 1 -- 172.21.15.90:0/2952334099 wait complete. 2024-06-02T15:56:10.237 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:56:10.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:56:10.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:10 smithi090 ceph-mon[31595]: pgmap v870: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:10.708 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:10 smithi120 ceph-mon[36451]: pgmap v870: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:10.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:10 smithi135 ceph-mon[36839]: pgmap v870: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:12.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:12 smithi090 ceph-mon[31595]: pgmap v871: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:12 smithi120 ceph-mon[36451]: pgmap v871: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:12.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:12 smithi135 ceph-mon[36839]: pgmap v871: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:14.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:14 smithi090 ceph-mon[31595]: pgmap v872: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 -- 172.21.15.90:0/2633102621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc10b650 msgr2=0x7f7dfc10ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 --2- 172.21.15.90:0/2633102621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc10b650 0x7f7dfc10ba30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009990 tx=0x7f7dec009ca0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 -- 172.21.15.90:0/2633102621 shutdown_connections 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 --2- 172.21.15.90:0/2633102621 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc106010 0x7f7dfc112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 --2- 172.21.15.90:0/2633102621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc10b650 0x7f7dfc10ba30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 --2- 172.21.15.90:0/2633102621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc105650 0x7f7dfc105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 -- 172.21.15.90:0/2633102621 >> 172.21.15.90:0/2633102621 conn(0x7f7dfc101120 msgr2=0x7f7dfc103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 -- 172.21.15.90:0/2633102621 shutdown_connections 2024-06-02T15:56:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.689+0000 7f7e01000700 1 -- 172.21.15.90:0/2633102621 wait complete. 2024-06-02T15:56:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.690+0000 7f7e01000700 1 Processor -- start 2024-06-02T15:56:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.690+0000 7f7e01000700 1 -- start start 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 0x7f7dfc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 0x7f7dfc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc10b650 0x7f7dfc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc115830 con 0x7f7dfc105650 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1156b0 con 0x7f7dfc10b650 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7e01000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1159b0 con 0x7f7dfc106010 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7dfb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 0x7f7dfc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7df3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 0x7f7dfc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7dfb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 0x7f7dfc19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51478/0 (socket says 172.21.15.90:51478) 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7dfb400700 1 -- 172.21.15.90:0/51656029 learned_addr learned my addr 172.21.15.90:0/51656029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.691+0000 7f7dfbe00700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc10b650 0x7f7dfc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 -- 172.21.15.90:0/51656029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc10b650 msgr2=0x7f7dfc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc10b650 0x7f7dfc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 -- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 msgr2=0x7f7dfc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 0x7f7dfc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 -- 172.21.15.90:0/51656029 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc1a0d30 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df3e00700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 0x7f7dfc19c260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e9f0 tx=0x7f7de400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400c9c0 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de400cb60 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4018680 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc1a1020 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.692+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a88a0 con 0x7f7dfc106010 2024-06-02T15:56:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.694+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc10dbe0 con 0x7f7dfc106010 2024-06-02T15:56:14.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.696+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7de4010640 con 0x7f7dfc106010 2024-06-02T15:56:14.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.698+0000 7f7df8c00700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 0x7f7de00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:14.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.698+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f7de40e2c40 con 0x7f7dfc106010 2024-06-02T15:56:14.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.698+0000 7f7dfb400700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 0x7f7de00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:14.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.699+0000 7f7dfb400700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 0x7f7de00b4dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005cd0 tx=0x7f7dec005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:14.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.700+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7de40aa550 con 0x7f7dfc106010 2024-06-02T15:56:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:14 smithi120 ceph-mon[36451]: pgmap v872: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:14.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:14 smithi135 ceph-mon[36839]: pgmap v872: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.981+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dfc19d080 con 0x7f7dfc106010 2024-06-02T15:56:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.983+0000 7f7df8c00700 1 -- 172.21.15.90:0/51656029 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11889 (secure 0 0 0) 0x7f7de40a9e50 con 0x7f7dfc106010 2024-06-02T15:56:14.980 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:56:14.980 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:54:27.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24524,24536,24512]},"id":1}]} 2024-06-02T15:56:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.986+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 msgr2=0x7f7de00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.986+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 0x7f7de00b4dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005cd0 tx=0x7f7dec005c60 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.986+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 msgr2=0x7f7dfc19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.986+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 0x7f7dfc19c260 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e9f0 tx=0x7f7de400ed00 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 shutdown_connections 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f7de00b2900 0x7f7de00b4dc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7dfc10b650 0x7f7dfc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc105650 0x7f7dfc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 --2- 172.21.15.90:0/51656029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7dfc106010 0x7f7dfc19c260 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 >> 172.21.15.90:0/51656029 conn(0x7f7dfc101120 msgr2=0x7f7dfc101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 shutdown_connections 2024-06-02T15:56:14.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:14.987+0000 7f7e01000700 1 -- 172.21.15.90:0/51656029 wait complete. 2024-06-02T15:56:14.986 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-06-02T15:56:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:15 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/51656029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:15.717 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:15 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/51656029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:15.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:15 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/51656029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:16.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:56:16.586 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:16.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:16 smithi090 ceph-mon[31595]: pgmap v873: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:56:16.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:16 smithi120 ceph-mon[36451]: pgmap v873: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:56:16.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:16 smithi135 ceph-mon[36839]: pgmap v873: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 10 op/s 2024-06-02T15:56:18.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:18 smithi090 ceph-mon[31595]: pgmap v874: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:56:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:18 smithi120 ceph-mon[36451]: pgmap v874: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:56:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:18 smithi135 ceph-mon[36839]: pgmap v874: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-06-02T15:56:20.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:20 smithi090 ceph-mon[31595]: pgmap v875: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:20.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.689+0000 7f60af200700 1 -- 172.21.15.90:0/2847406980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b010e950 msgr2=0x7f60b010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.689+0000 7f60af200700 1 --2- 172.21.15.90:0/2847406980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b010e950 0x7f60b010ed30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009990 tx=0x7f60a0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 -- 172.21.15.90:0/2847406980 shutdown_connections 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 --2- 172.21.15.90:0/2847406980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010a440 0x7f60b01168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 --2- 172.21.15.90:0/2847406980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b010e950 0x7f60b010ed30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 --2- 172.21.15.90:0/2847406980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b0109a80 0x7f60b0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 -- 172.21.15.90:0/2847406980 >> 172.21.15.90:0/2847406980 conn(0x7f60b00792d0 msgr2=0x7f60b0108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 -- 172.21.15.90:0/2847406980 shutdown_connections 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.690+0000 7f60af200700 1 -- 172.21.15.90:0/2847406980 wait complete. 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.691+0000 7f60af200700 1 Processor -- start 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.691+0000 7f60af200700 1 -- start start 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.691+0000 7f60af200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 0x7f60b01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.691+0000 7f60af200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.691+0000 7f60af200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010e950 0x7f60b01a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60af200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0119c10 con 0x7f60b010a440 2024-06-02T15:56:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60af200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0119a90 con 0x7f60b0109a80 2024-06-02T15:56:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60af200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0119d90 con 0x7f60b010e950 2024-06-02T15:56:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51500/0 (socket says 172.21.15.90:51500) 2024-06-02T15:56:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ad400700 1 -- 172.21.15.90:0/2693050330 learned_addr learned my addr 172.21.15.90:0/2693050330 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 0x7f60b01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ae800700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010e950 0x7f60b01a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 -- 172.21.15.90:0/2693050330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010e950 msgr2=0x7f60b01a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010e950 0x7f60b01a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 -- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 msgr2=0x7f60b01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ade00700 1 -- 172.21.15.90:0/2693050330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b01a50e0 con 0x7f60b0109a80 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.692+0000 7f60ad400700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60ade00700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 0x7f60b01a0120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60a6a00700 1 -- 172.21.15.90:0/2693050330 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0027070 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60a6a00700 1 -- 172.21.15.90:0/2693050330 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a0022470 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60a6a00700 1 -- 172.21.15.90:0/2693050330 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0017650 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b01a5370 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.693+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b01acc50 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.697+0000 7f60a6a00700 1 -- 172.21.15.90:0/2693050330 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f60a002a070 con 0x7f60b0109a80 2024-06-02T15:56:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.698+0000 7f60bf52eb80 1 Processor -- start 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.698+0000 7f60bf52eb80 1 -- start start 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.698+0000 7f60a6a00700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 0x7f60940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60a6a00700 1 -- 172.21.15.90:0/2693050330 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f60a00e3200 con 0x7f60b0109a80 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16be9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ae800700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 0x7f60940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557f16b47e40 con 0x557f16bf00e0 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f16b379e0 con 0x557f16bef7c0 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f16b3ba20 con 0x557f16bea390 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16be9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:57774/0 (socket says 172.21.15.90:57774) 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16be9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51510/0 (socket says 172.21.15.90:51510) 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ade00700 1 -- 172.21.15.90:0/657974880 learned_addr learned my addr 172.21.15.90:0/657974880 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.699+0000 7f60ae800700 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 -- 172.21.15.90:0/657974880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 msgr2=0x557f16bea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 -- 172.21.15.90:0/657974880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 msgr2=0x557f16be9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16be9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 -- 172.21.15.90:0/657974880 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f16b00420 con 0x557f16bef7c0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ae800700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 0x7f60940b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009ec0 tx=0x7f60a800fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f60ade00700 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f60a00aa1a0 tx=0x7f60a00a9e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d2791396cf6801 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f609b400700 1 -- 172.21.15.90:0/657974880 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00a9fb0 con 0x557f16bef7c0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f609b400700 1 -- 172.21.15.90:0/657974880 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00aaba0 con 0x557f16bef7c0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.700+0000 7f609b400700 1 -- 172.21.15.90:0/657974880 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00aa740 con 0x557f16bef7c0 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 -- 172.21.15.90:0/657974880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 msgr2=0x557f16befba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f60a00aa1a0 tx=0x7f60a00a9e70 comp rx=0 tx=0).stop 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 -- 172.21.15.90:0/657974880 shutdown_connections 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bef7c0 0x557f16befba0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/657974880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16be9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 -- 172.21.15.90:0/657974880 >> 172.21.15.90:0/657974880 conn(0x557f16b17be0 msgr2=0x557f16be4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 -- 172.21.15.90:0/657974880 shutdown_connections 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.701+0000 7f60bf52eb80 1 -- 172.21.15.90:0/657974880 wait complete. 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.702+0000 7f60bf52eb80 1 -- start start 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.702+0000 7f60bf52eb80 1 Processor -- start 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bf5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bef7c0 0x557f16bf31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557f16a3dfa0 con 0x557f16bef7c0 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x557f16ad8760 con 0x557f16bf00e0 2024-06-02T15:56:20.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557f16bf69c0 con 0x557f16bea390 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60bf52eb80 10 client.0 _ll_register_callbacks cb 0x557f16ae1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60ad400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bef7c0 0x557f16bf31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60ae800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60ade00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bf5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60ae800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:57782/0 (socket says 172.21.15.90:57782) 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.703+0000 7f60ae800700 1 -- 172.21.15.90:0/1952923939 learned_addr learned my addr 172.21.15.90:0/1952923939 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 -- 172.21.15.90:0/1952923939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 msgr2=0x557f16bf5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bf5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 -- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bef7c0 msgr2=0x557f16bf31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bef7c0 0x557f16bf31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 -- 172.21.15.90:0/1952923939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f16a878d0 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60ae800700 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60a8018dd0 tx=0x7f60a8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2686d6e0231affd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f608fe00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8008c80 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f60a8003a00 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.704+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a00aaba0 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f60a8032050 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 10 client.26882 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f16b08060 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a00aaba0 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f60a8004600 con 0x557f16bf00e0 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 10 client.26882 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 10 client.26882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 20 client.26882 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c03590 0x557f16c05980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 4 client.26882 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.705+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f16b49d90 con 0x557f16c03590 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.706+0000 7f60ade00700 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c03590 0x557f16c05980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.706+0000 7f60ade00700 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c03590 0x557f16c05980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69841b8f1a4bd8d0 server_cookie=677e3d7083f0066c in_seq=0 out_seq=0 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.706+0000 7f608fe00700 10 client.26882 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f608fe00700 1 -- 172.21.15.90:0/1952923939 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557f16b49d90 con 0x557f16c03590 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f608fe00700 10 client.26882 handle_command_reply: tid=0 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 client.26882 shutdown 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 msgr2=0x557f16bf36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f60a8018dd0 tx=0x7f60a8004b40 comp rx=0 tx=0).stop 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 shutdown_connections 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c03590 0x557f16c05980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.707+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bf00e0 0x557f16bf36e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f608fe00700 0 client.26882 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bea390 0x557f16bf5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1952923939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bef7c0 0x557f16bf31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 >> 172.21.15.90:0/1952923939 conn(0x557f16b17be0 msgr2=0x557f16be3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 shutdown_connections 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1952923939 wait complete. 2024-06-02T15:56:20.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.708+0000 7f60bf52eb80 20 client.26882 trim_cache size 0 max 16384 2024-06-02T15:56:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 Processor -- start 2024-06-02T15:56:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 -- start start 2024-06-02T15:56:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16be4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b80b0 con 0x557f16bf00e0 2024-06-02T15:56:20.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b5880 con 0x557f16bea390 2024-06-02T15:56:20.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.725+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b36d0 con 0x557f16bef7c0 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ae800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51518/0 (socket says 172.21.15.90:51518) 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:57790/0 (socket says 172.21.15.90:57790) 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ae800700 1 -- 172.21.15.90:0/1753224089 learned_addr learned my addr 172.21.15.90:0/1753224089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ad400700 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16be4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 -- 172.21.15.90:0/1753224089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 msgr2=0x557f16be4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16be4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 -- 172.21.15.90:0/1753224089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 msgr2=0x557f16c0b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.726+0000 7f60ade00700 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f60ade00700 1 -- 172.21.15.90:0/1753224089 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f16c05cf0 con 0x557f16bea390 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f60ae800700 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f60ade00700 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b680 tx=0x7f60a0092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d488f845872bd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f609aa00700 1 -- 172.21.15.90:0/1753224089 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00ae850 con 0x557f16bea390 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f609aa00700 1 -- 172.21.15.90:0/1753224089 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a0022940 con 0x557f16bea390 2024-06-02T15:56:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.727+0000 7f609aa00700 1 -- 172.21.15.90:0/1753224089 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a00b3040 con 0x557f16bea390 2024-06-02T15:56:20.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1753224089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 msgr2=0x557f16be9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b680 tx=0x7f60a0092040 comp rx=0 tx=0).stop 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1753224089 shutdown_connections 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16be9d70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/1753224089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16be4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1753224089 >> 172.21.15.90:0/1753224089 conn(0x557f16b17be0 msgr2=0x557f16bf1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1753224089 shutdown_connections 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.728+0000 7f60bf52eb80 1 -- 172.21.15.90:0/1753224089 wait complete. 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.729+0000 7f60bf52eb80 1 -- start start 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.729+0000 7f60bf52eb80 1 Processor -- start 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16bfbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b3d40 con 0x557f16bf00e0 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b51b0 con 0x557f16bea390 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01b4410 con 0x557f16bef7c0 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60bf52eb80 10 client.0 _ll_register_callbacks cb 0x557f16ae1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ade00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16bfbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:53670/0 (socket says 172.21.15.90:53670) 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 -- 172.21.15.90:0/772452787 learned_addr learned my addr 172.21.15.90:0/772452787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:20.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ae800700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 -- 172.21.15.90:0/772452787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 msgr2=0x557f16bfbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16bfbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 -- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 msgr2=0x557f16c0aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ad400700 1 -- 172.21.15.90:0/772452787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f16c0b230 con 0x557f16bef7c0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.730+0000 7f60ade00700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16bfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f60ad400700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f609c0096b0 tx=0x7f609c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542e1543ea354e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f608fe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c023060 con 0x557f16bef7c0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a0022940 con 0x557f16bef7c0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f609c00fd50 con 0x557f16bef7c0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f609c02a050 con 0x557f16bef7c0 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f60bf52eb80 10 client.26821 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:20.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.731+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f16c6de00 con 0x557f16bef7c0 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a0022940 con 0x557f16bef7c0 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f609c003c90 con 0x557f16bef7c0 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 10 client.26821 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 10 client.26821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 20 client.26821 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c6e580 0x557f16c709a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 4 client.26821 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f16b43570 con 0x557f16c6e580 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.732+0000 7f60ade00700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c6e580 0x557f16c709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:20.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60ade00700 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c6e580 0x557f16c709a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=577b96e587f2b0e server_cookie=a6b38ac98fedf2ee in_seq=0 out_seq=0 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f608fe00700 10 client.26821 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f608fe00700 1 -- 172.21.15.90:0/772452787 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f16b43570 con 0x557f16c6e580 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f608fe00700 10 client.26821 handle_command_reply: tid=0 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 client.26821 shutdown 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 msgr2=0x557f16bfc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f609c0096b0 tx=0x7f609c00f6a0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 shutdown_connections 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x557f16c6e580 0x557f16c709a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557f16bf00e0 0x557f16c0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x557f16bea390 0x557f16bfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f608fe00700 0 client.26821 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.733+0000 7f60bf52eb80 1 --2- 172.21.15.90:0/772452787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557f16bef7c0 0x557f16bfc430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.734+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 >> 172.21.15.90:0/772452787 conn(0x557f16b17be0 msgr2=0x557f16bf04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.734+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 shutdown_connections 2024-06-02T15:56:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.734+0000 7f60bf52eb80 1 -- 172.21.15.90:0/772452787 wait complete. 2024-06-02T15:56:20.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.734+0000 7f60bf52eb80 20 client.26821 trim_cache size 0 max 16384 2024-06-02T15:56:20.734 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 msgr2=0x7f60940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 0x7f60940b4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f60a8009ec0 tx=0x7f60a800fc40 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 msgr2=0x7f60b01a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 0x7f60b01a0120 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f60a0006010 tx=0x7f60a00057c0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 shutdown_connections 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f60940b2880 0x7f60940b4d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60b010e950 0x7f60b01a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f60b0109a80 0x7f60b01a0120 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 --2- 172.21.15.90:0/2693050330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60b010a440 0x7f60b01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:20.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 >> 172.21.15.90:0/2693050330 conn(0x7f60b00792d0 msgr2=0x7f60b0079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:20.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 shutdown_connections 2024-06-02T15:56:20.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:20.735+0000 7f60af200700 1 -- 172.21.15.90:0/2693050330 wait complete. 2024-06-02T15:56:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:20 smithi120 ceph-mon[36451]: pgmap v875: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:20 smithi135 ceph-mon[36839]: pgmap v875: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:22.022 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:56:22.474 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:22.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:22 smithi090 ceph-mon[31595]: pgmap v876: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:22 smithi120 ceph-mon[36451]: pgmap v876: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:22 smithi135 ceph-mon[36839]: pgmap v876: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:24.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:24 smithi090 ceph-mon[31595]: pgmap v877: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:24 smithi120 ceph-mon[36451]: pgmap v877: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:24 smithi135 ceph-mon[36839]: pgmap v877: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.311+0000 7f3cafe00700 1 -- 172.21.15.90:0/2141409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb010dab0 msgr2=0x7f3cb010ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.311+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2141409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb010dab0 0x7f3cb010ff70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b0a0 tx=0x7f3ca800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.311+0000 7f3cafe00700 1 -- 172.21.15.90:0/2141409160 shutdown_connections 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.311+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2141409160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb010dab0 0x7f3cb010ff70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.311+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2141409160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb0104680 0x7f3cb0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.312+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2141409160 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104fa0 0x7f3cb010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.312+0000 7f3cafe00700 1 -- 172.21.15.90:0/2141409160 >> 172.21.15.90:0/2141409160 conn(0x7f3cb00fded0 msgr2=0x7f3cb01002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.312+0000 7f3cafe00700 1 -- 172.21.15.90:0/2141409160 shutdown_connections 2024-06-02T15:56:26.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.312+0000 7f3cafe00700 1 -- 172.21.15.90:0/2141409160 wait complete. 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.313+0000 7f3cafe00700 1 Processor -- start 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.313+0000 7f3cafe00700 1 -- start start 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb0104fa0 0x7f3cb019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb010dab0 0x7f3cb0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb0113720 con 0x7f3cb010dab0 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01135a0 con 0x7f3cb0104680 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cafe00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01138a0 con 0x7f3cb0104fa0 2024-06-02T15:56:26.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3cae000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb0104fa0 0x7f3cb019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59620/0 (socket says 172.21.15.90:59620) 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 -- 172.21.15.90:0/2741862874 learned_addr learned my addr 172.21.15.90:0/2741862874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caf400700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb010dab0 0x7f3cb0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 -- 172.21.15.90:0/2741862874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb0104fa0 msgr2=0x7f3cb019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb0104fa0 0x7f3cb019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 -- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb010dab0 msgr2=0x7f3cb0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb010dab0 0x7f3cb0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.314+0000 7f3caea00700 1 -- 172.21.15.90:0/2741862874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb019a240 con 0x7f3cb0104680 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.315+0000 7f3caea00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9a0 tx=0x7f3ca000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.316+0000 7f3ca7400700 1 -- 172.21.15.90:0/2741862874 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000ca10 con 0x7f3cb0104680 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.316+0000 7f3ca7400700 1 -- 172.21.15.90:0/2741862874 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca000cbb0 con 0x7f3cb0104680 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.316+0000 7f3ca7400700 1 -- 172.21.15.90:0/2741862874 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0018680 con 0x7f3cb0104680 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.316+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb019a530 con 0x7f3cb0104680 2024-06-02T15:56:26.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.317+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb01a67a0 con 0x7f3cb0104680 2024-06-02T15:56:26.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.320+0000 7f3ca7400700 1 -- 172.21.15.90:0/2741862874 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3ca0010430 con 0x7f3cb0104680 2024-06-02T15:56:26.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.321+0000 7f3cc004cb80 1 Processor -- start 2024-06-02T15:56:26.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.321+0000 7f3ca7400700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 0x7f3c940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.321+0000 7f3cc004cb80 1 -- start start 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3ca7400700 1 -- 172.21.15.90:0/2741862874 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca00e2710 con 0x7f3cb0104680 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3caf400700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 0x7f3c940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 0x561ea8f3ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 0x561ea8f3b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f3bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ea8e94e40 con 0x561ea8f3fba0 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561ea8e849e0 con 0x561ea8f404c0 2024-06-02T15:56:26.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561ea8e88a20 con 0x561ea8f3b7a0 2024-06-02T15:56:26.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 0x561ea8f3ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 0x561ea8f3ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44924/0 (socket says 172.21.15.90:44924) 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3caea00700 1 -- 172.21.15.90:0/1496054674 learned_addr learned my addr 172.21.15.90:0/1496054674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.322+0000 7f3cae000700 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 0x561ea8f3b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3caf400700 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f3bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 -- 172.21.15.90:0/1496054674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 msgr2=0x561ea8f3bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f3bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 -- 172.21.15.90:0/1496054674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 msgr2=0x561ea8f3ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 0x561ea8f3ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3caf400700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 0x7f3c940b4dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b780 tx=0x7f3ca8006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 -- 172.21.15.90:0/1496054674 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea8e4d420 con 0x561ea8f404c0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3cae000700 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 0x561ea8f3b180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0105400 tx=0x7f3c9c009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2374b8c4fb1c21 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.323+0000 7f3c9be00700 1 -- 172.21.15.90:0/1496054674 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c011c50 con 0x561ea8f404c0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.324+0000 7f3c9be00700 1 -- 172.21.15.90:0/1496054674 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9c011df0 con 0x561ea8f404c0 2024-06-02T15:56:26.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.324+0000 7f3c9be00700 1 -- 172.21.15.90:0/1496054674 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c005280 con 0x561ea8f404c0 2024-06-02T15:56:26.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.324+0000 7f3cc004cb80 1 -- 172.21.15.90:0/1496054674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 msgr2=0x561ea8f3b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.324+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 0x561ea8f3b180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0105400 tx=0x7f3c9c009f30 comp rx=0 tx=0).stop 2024-06-02T15:56:26.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 -- 172.21.15.90:0/1496054674 shutdown_connections 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f3bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3fba0 0x561ea8f3ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/1496054674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f404c0 0x561ea8f3b180 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 -- 172.21.15.90:0/1496054674 >> 172.21.15.90:0/1496054674 conn(0x561ea8e64be0 msgr2=0x561ea8f31810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 -- 172.21.15.90:0/1496054674 shutdown_connections 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 -- 172.21.15.90:0/1496054674 wait complete. 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 -- start start 2024-06-02T15:56:26.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.325+0000 7f3cc004cb80 1 Processor -- start 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.326+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.326+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3fba0 0x561ea8f44610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f44b50 0x561ea8f44fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ea8d8afa0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x561ea8e25760 con 0x561ea8f44b50 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561ea8f48040 con 0x561ea8f3fba0 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3cc004cb80 10 client.0 _ll_register_callbacks cb 0x561ea8e2ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44926/0 (socket says 172.21.15.90:44926) 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 -- 172.21.15.90:0/838980820 learned_addr learned my addr 172.21.15.90:0/838980820 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:26.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caf400700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f44b50 0x561ea8f44fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 -- 172.21.15.90:0/838980820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3fba0 msgr2=0x561ea8f44610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3fba0 0x561ea8f44610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 -- 172.21.15.90:0/838980820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f44b50 msgr2=0x561ea8f44fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f44b50 0x561ea8f44fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.327+0000 7f3caea00700 1 -- 172.21.15.90:0/838980820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea8dd48d0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3caea00700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00098c0 tx=0x7f3ca00cc330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3394e4c21daa88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3c98c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00c3060 con 0x561ea8f3b7a0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca00cabd0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9c011df0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ca00aa020 con 0x561ea8f3b7a0 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3cc004cb80 10 client.16602 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.328+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ea8e55060 con 0x561ea8f3b7a0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9c011df0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3ca00b22f0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 10 client.16602 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 10 client.16602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 20 client.16602 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8f54fc0 0x561ea8f57430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 4 client.16602 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea8e96d90 con 0x561ea8f54fc0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.329+0000 7f3cae000700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8f54fc0 0x561ea8f57430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.330+0000 7f3c98c00700 10 client.16602 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.330+0000 7f3cae000700 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8f54fc0 0x561ea8f57430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c0f25895c98a1de server_cookie=346f360710cbec1b in_seq=0 out_seq=0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.331+0000 7f3c98c00700 1 -- 172.21.15.90:0/838980820 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561ea8e96d90 con 0x561ea8f54fc0 2024-06-02T15:56:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.331+0000 7f3c98c00700 10 client.16602 handle_command_reply: tid=0 2024-06-02T15:56:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.331+0000 7f3cc004cb80 1 client.16602 shutdown 2024-06-02T15:56:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.331+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 msgr2=0x561ea8f47270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00098c0 tx=0x7f3ca00cc330 comp rx=0 tx=0).stop 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 shutdown_connections 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8f54fc0 0x561ea8f57430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f44b50 0x561ea8f44fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f47270 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/838980820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3fba0 0x561ea8f44610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3c98c00700 0 client.16602 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 >> 172.21.15.90:0/838980820 conn(0x561ea8e64be0 msgr2=0x561ea8f30fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 shutdown_connections 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 1 -- 172.21.15.90:0/838980820 wait complete. 2024-06-02T15:56:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.332+0000 7f3cc004cb80 20 client.16602 trim_cache size 0 max 16384 2024-06-02T15:56:26.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.348+0000 7f3cc004cb80 1 Processor -- start 2024-06-02T15:56:26.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 -- start start 2024-06-02T15:56:26.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f3d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f44b50 0x561ea8f3dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01afa70 con 0x561ea8f3b7a0 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01ad240 con 0x561ea8f3fba0 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01ab090 con 0x561ea8f44b50 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3caf400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f44b50 0x561ea8f3dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3cae000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f3d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3caea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44930/0 (socket says 172.21.15.90:44930) 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.349+0000 7f3caea00700 1 -- 172.21.15.90:0/120147112 learned_addr learned my addr 172.21.15.90:0/120147112 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 -- 172.21.15.90:0/120147112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f44b50 msgr2=0x561ea8f3dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f44b50 0x561ea8f3dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 -- 172.21.15.90:0/120147112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 msgr2=0x561ea8f31740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 -- 172.21.15.90:0/120147112 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea8f42b80 con 0x561ea8f3fba0 2024-06-02T15:56:26.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3caea00700 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3cae000700 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f3d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x561ea8f32690 tx=0x7f3c9c004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14913d2444473ac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3c9b400700 1 -- 172.21.15.90:0/120147112 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c02b040 con 0x561ea8f3fba0 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3c9b400700 1 -- 172.21.15.90:0/120147112 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c9c005d00 con 0x561ea8f3fba0 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.350+0000 7f3c9b400700 1 -- 172.21.15.90:0/120147112 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c005050 con 0x561ea8f3fba0 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 -- 172.21.15.90:0/120147112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 msgr2=0x561ea8f3d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f3d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x561ea8f32690 tx=0x7f3c9c004d80 comp rx=0 tx=0).stop 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 -- 172.21.15.90:0/120147112 shutdown_connections 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f44b50 0x561ea8f3dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f3b7a0 0x561ea8f31740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/120147112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f3d6f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 -- 172.21.15.90:0/120147112 >> 172.21.15.90:0/120147112 conn(0x561ea8e64be0 msgr2=0x561ea8f443f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.351+0000 7f3cc004cb80 1 -- 172.21.15.90:0/120147112 shutdown_connections 2024-06-02T15:56:26.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.352+0000 7f3cc004cb80 1 -- 172.21.15.90:0/120147112 wait complete. 2024-06-02T15:56:26.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.352+0000 7f3cc004cb80 1 -- start start 2024-06-02T15:56:26.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.352+0000 7f3cc004cb80 1 Processor -- start 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f4d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f44b50 0x561ea8f41920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01ab700 con 0x561ea8f44b50 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01acb70 con 0x561ea8f3fba0 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb01abdd0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cc004cb80 10 client.0 _ll_register_callbacks cb 0x561ea8e2ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3caea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3caea00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:41856/0 (socket says 172.21.15.90:41856) 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3caea00700 1 -- 172.21.15.90:0/318655112 learned_addr learned my addr 172.21.15.90:0/318655112 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:26.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3cae000700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f4d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.353+0000 7f3caf400700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f44b50 0x561ea8f41920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 -- 172.21.15.90:0/318655112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 msgr2=0x561ea8f4d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f4d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 -- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f44b50 msgr2=0x561ea8f41920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f44b50 0x561ea8f41920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 -- 172.21.15.90:0/318655112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea8f42160 con 0x561ea8f3b7a0 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3caea00700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00caf00 tx=0x7f3ca00b21d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d5fe86fd1b607e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.354+0000 7f3c98c00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00adcd0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3ca00a8b80 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ea8e90570 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ca00b2780 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3cc004cb80 10 client.26836 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.358+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561ea8f5ccb0 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561ea8e90570 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7f3ca008a020 con 0x561ea8f3b7a0 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 10 client.26836 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 10 client.26836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 20 client.26836 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8fbfd40 0x561ea8fc2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 4 client.26836 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:56:26.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ea8d07250 con 0x561ea8fbfd40 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.359+0000 7f3cae000700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8fbfd40 0x561ea8fc2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cae000700 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8fbfd40 0x561ea8fc2160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb5ff77ce565e23c server_cookie=31e8d31a91544ba7 in_seq=0 out_seq=0 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3c98c00700 10 client.26836 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3c98c00700 1 -- 172.21.15.90:0/318655112 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ea8d07250 con 0x561ea8fbfd40 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3c98c00700 10 client.26836 handle_command_reply: tid=0 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 client.26836 shutdown 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 msgr2=0x561ea8f4d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00caf00 tx=0x7f3ca00b21d0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 shutdown_connections 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea8f44b50 0x561ea8f41920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ea8f3b7a0 0x561ea8f4d4b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x561ea8fbfd40 0x561ea8fc2160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 --2- 172.21.15.90:0/318655112 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x561ea8f3fba0 0x561ea8f4d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 >> 172.21.15.90:0/318655112 conn(0x561ea8e64be0 msgr2=0x561ea8f35d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.360+0000 7f3c98c00700 0 client.26836 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.361+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 shutdown_connections 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.361+0000 7f3cc004cb80 1 -- 172.21.15.90:0/318655112 wait complete. 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.361+0000 7f3cc004cb80 20 client.26836 trim_cache size 0 max 16384 2024-06-02T15:56:26.359 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 msgr2=0x7f3c940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 0x7f3c940b4dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800b780 tx=0x7f3ca8006210 comp rx=0 tx=0).stop 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 msgr2=0x7f3cb019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9a0 tx=0x7f3ca000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 shutdown_connections 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f3c940b2900 0x7f3c940b4dc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cb010dab0 0x7f3cb0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3cb0104680 0x7f3cb019f8d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 --2- 172.21.15.90:0/2741862874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3cb0104fa0 0x7f3cb019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:26.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 >> 172.21.15.90:0/2741862874 conn(0x7f3cb00fded0 msgr2=0x7f3cb00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:26.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 shutdown_connections 2024-06-02T15:56:26.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:26.362+0000 7f3cafe00700 1 -- 172.21.15.90:0/2741862874 wait complete. 2024-06-02T15:56:26.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:26 smithi090 ceph-mon[31595]: pgmap v878: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:26.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:26 smithi120 ceph-mon[36451]: pgmap v878: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:26.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:26 smithi135 ceph-mon[36839]: pgmap v878: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:27.324 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:56:27.806 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:28.609 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:28 smithi120 ceph-mon[36451]: pgmap v879: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:28.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:28 smithi090 ceph-mon[31595]: pgmap v879: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:28.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:28 smithi135 ceph-mon[36839]: pgmap v879: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:30.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:30 smithi090 ceph-mon[31595]: pgmap v880: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:30.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:30 smithi120 ceph-mon[36451]: pgmap v880: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:30.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:30 smithi135 ceph-mon[36839]: pgmap v880: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:56:32.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 -- 172.21.15.90:0/2149077610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 msgr2=0x7fae3c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 --2- 172.21.15.90:0/2149077610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c104a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae30009990 tx=0x7fae30009ca0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 -- 172.21.15.90:0/2149077610 shutdown_connections 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 --2- 172.21.15.90:0/2149077610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c10da60 0x7fae3c10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 --2- 172.21.15.90:0/2149077610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c104a10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 --2- 172.21.15.90:0/2149077610 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c104f50 0x7fae3c10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 -- 172.21.15.90:0/2149077610 >> 172.21.15.90:0/2149077610 conn(0x7fae3c0fde60 msgr2=0x7fae3c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 -- 172.21.15.90:0/2149077610 shutdown_connections 2024-06-02T15:56:32.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.195+0000 7fae42a00700 1 -- 172.21.15.90:0/2149077610 wait complete. 2024-06-02T15:56:32.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.196+0000 7fae42a00700 1 Processor -- start 2024-06-02T15:56:32.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.196+0000 7fae42a00700 1 -- start start 2024-06-02T15:56:32.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.196+0000 7fae42a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.196+0000 7fae42a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c104f50 0x7fae3c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.196+0000 7fae42a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c10da60 0x7fae3c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae42a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1136a0 con 0x7fae3c104630 2024-06-02T15:56:32.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae42a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c113520 con 0x7fae3c10da60 2024-06-02T15:56:32.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae42a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c113820 con 0x7fae3c104f50 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49282/0 (socket says 172.21.15.90:49282) 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 -- 172.21.15.90:0/746696075 learned_addr learned my addr 172.21.15.90:0/746696075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae40c00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c104f50 0x7fae3c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae42000700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c10da60 0x7fae3c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 -- 172.21.15.90:0/746696075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c104f50 msgr2=0x7fae3c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c104f50 0x7fae3c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 -- 172.21.15.90:0/746696075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c10da60 msgr2=0x7fae3c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c10da60 0x7fae3c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.197+0000 7fae41600700 1 -- 172.21.15.90:0/746696075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c19a1e0 con 0x7fae3c104630 2024-06-02T15:56:32.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae41600700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae300057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae3a000700 1 -- 172.21.15.90:0/746696075 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30027070 con 0x7fae3c104630 2024-06-02T15:56:32.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae3a000700 1 -- 172.21.15.90:0/746696075 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae30022470 con 0x7fae3c104630 2024-06-02T15:56:32.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c19a470 con 0x7fae3c104630 2024-06-02T15:56:32.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c1021c0 con 0x7fae3c104630 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.198+0000 7fae3a000700 1 -- 172.21.15.90:0/746696075 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30017650 con 0x7fae3c104630 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.201+0000 7fae3a000700 1 -- 172.21.15.90:0/746696075 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae3002a070 con 0x7fae3c104630 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 Processor -- start 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 -- start start 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae3a000700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 0x7fae240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c58400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae3a000700 1 -- 172.21.15.90:0/746696075 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae300e3350 con 0x7fae3c104630 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c52f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5584d5bb0e40 con 0x5584d5c58940 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584d5ba09e0 con 0x5584d5c58020 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5584d5ba4a20 con 0x5584d5c52ad0 2024-06-02T15:56:32.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.203+0000 7fae40c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae42000700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 0x7fae240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae41600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c58400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae41600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c58400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59228/0 (socket says 172.21.15.90:59228) 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae41600700 1 -- 172.21.15.90:0/1288091024 learned_addr learned my addr 172.21.15.90:0/1288091024 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae42000700 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c52f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae40c00700 1 -- 172.21.15.90:0/1288091024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 msgr2=0x5584d5c52f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae40c00700 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c52f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae40c00700 1 -- 172.21.15.90:0/1288091024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 msgr2=0x5584d5c58400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae40c00700 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c58400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae40c00700 1 -- 172.21.15.90:0/1288091024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d5b69420 con 0x5584d5c58940 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.204+0000 7fae42000700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 0x7fae240b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae34009ca0 tx=0x7fae34003a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.205+0000 7fae40c00700 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c524b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fae3c1053b0 tx=0x7fae28009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc0efd2a080653e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.205+0000 7fae2ea00700 1 -- 172.21.15.90:0/1288091024 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28011cb0 con 0x5584d5c58940 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.205+0000 7fae2ea00700 1 -- 172.21.15.90:0/1288091024 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae28011e50 con 0x5584d5c58940 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.205+0000 7fae2ea00700 1 -- 172.21.15.90:0/1288091024 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28004e60 con 0x5584d5c58940 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1288091024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 msgr2=0x5584d5c524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c524b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fae3c1053b0 tx=0x7fae28009f50 comp rx=0 tx=0).stop 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1288091024 shutdown_connections 2024-06-02T15:56:32.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c52f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c58400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1288091024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c524b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1288091024 >> 172.21.15.90:0/1288091024 conn(0x5584d5b80be0 msgr2=0x5584d5c4d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1288091024 shutdown_connections 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1288091024 wait complete. 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.206+0000 7fae4ea27b80 1 -- start start 2024-06-02T15:56:32.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 Processor -- start 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c58020 0x5584d5c5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5584d5aa6fa0 con 0x5584d5c58940 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5584d5b41760 con 0x5584d5c52ad0 2024-06-02T15:56:32.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5584d5c5f460 con 0x5584d5c58020 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae4ea27b80 10 client.0 _ll_register_callbacks cb 0x5584d5b4ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.207+0000 7fae41600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae42000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59240/0 (socket says 172.21.15.90:59240) 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 -- 172.21.15.90:0/1444007408 learned_addr learned my addr 172.21.15.90:0/1444007408 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae40c00700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c58020 0x5584d5c5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 -- 172.21.15.90:0/1444007408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c58020 msgr2=0x5584d5c5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c58020 0x5584d5c5ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 -- 172.21.15.90:0/1444007408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 msgr2=0x5584d5c5bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 -- 172.21.15.90:0/1444007408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d5af08d0 con 0x5584d5c52ad0 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae41600700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fae300a6670 tx=0x7fae300a9d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2f427034a9d91e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae23400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:32.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300aa5f0 con 0x5584d5c52ad0 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.208+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae28011e50 con 0x5584d5c52ad0 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae300aa850 con 0x5584d5c52ad0 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae300ae9a0 con 0x5584d5c52ad0 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 10 client.26918 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5584d5b71060 con 0x5584d5c52ad0 2024-06-02T15:56:32.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae28011e50 con 0x5584d5c52ad0 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fae30005110 con 0x5584d5c52ad0 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 20 client.26918 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5c6be70 0x5584d5c6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 4 client.26918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.209+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d5bb2d90 con 0x5584d5c6be70 2024-06-02T15:56:32.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.210+0000 7fae40c00700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5c6be70 0x5584d5c6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.210+0000 7fae40c00700 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5c6be70 0x5584d5c6e2a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2a2f86d887a9e5 server_cookie=130a5d6a4eccd578 in_seq=0 out_seq=0 2024-06-02T15:56:32.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.210+0000 7fae23400700 10 client.26918 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae23400700 1 -- 172.21.15.90:0/1444007408 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5584d5bb2d90 con 0x5584d5c6be70 2024-06-02T15:56:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae23400700 10 client.26918 handle_command_reply: tid=0 2024-06-02T15:56:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 client.26918 shutdown 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 msgr2=0x5584d5c5e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fae300a6670 tx=0x7fae300a9d60 comp rx=0 tx=0).stop 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 shutdown_connections 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5c6be70 0x5584d5c6e2a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c52ad0 0x5584d5c5e670 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/1444007408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c58020 0x5584d5c5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.211+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 >> 172.21.15.90:0/1444007408 conn(0x5584d5b80be0 msgr2=0x5584d5c514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.212+0000 7fae23400700 0 client.26918 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.212+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 shutdown_connections 2024-06-02T15:56:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.212+0000 7fae4ea27b80 1 -- 172.21.15.90:0/1444007408 wait complete. 2024-06-02T15:56:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.212+0000 7fae4ea27b80 20 client.26918 trim_cache size 0 max 16384 2024-06-02T15:56:32.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 Processor -- start 2024-06-02T15:56:32.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 -- start start 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1b3d40 con 0x5584d5c58940 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1b1510 con 0x5584d5c58020 2024-06-02T15:56:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.228+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1af360 con 0x5584d5c52ad0 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae40c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49302/0 (socket says 172.21.15.90:49302) 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae41600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 -- 172.21.15.90:0/2652560486 learned_addr learned my addr 172.21.15.90:0/2652560486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae40c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59248/0 (socket says 172.21.15.90:59248) 2024-06-02T15:56:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 -- 172.21.15.90:0/2652560486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 msgr2=0x5584d5c524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 -- 172.21.15.90:0/2652560486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 msgr2=0x5584d5c47a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae42000700 1 -- 172.21.15.90:0/2652560486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d5c59f80 con 0x5584d5c58940 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.229+0000 7fae40c00700 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:32.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.230+0000 7fae42000700 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fae34009fa0 tx=0x7fae3401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a5d81b29bfb08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.230+0000 7fae2e000700 1 -- 172.21.15.90:0/2652560486 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3402a070 con 0x5584d5c58940 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.230+0000 7fae2e000700 1 -- 172.21.15.90:0/2652560486 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3401a920 con 0x5584d5c58940 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.230+0000 7fae2e000700 1 -- 172.21.15.90:0/2652560486 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34025860 con 0x5584d5c58940 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- 172.21.15.90:0/2652560486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 msgr2=0x5584d5c5a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fae34009fa0 tx=0x7fae3401a000 comp rx=0 tx=0).stop 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- 172.21.15.90:0/2652560486 shutdown_connections 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58940 0x5584d5c5a300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/2652560486 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58020 0x5584d5c47a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- 172.21.15.90:0/2652560486 >> 172.21.15.90:0/2652560486 conn(0x5584d5b80be0 msgr2=0x5584d5c55aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- 172.21.15.90:0/2652560486 shutdown_connections 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- 172.21.15.90:0/2652560486 wait complete. 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 -- start start 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.231+0000 7fae4ea27b80 1 Processor -- start 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 0x5584d5c737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1af9d0 con 0x5584d5c58020 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1b0e40 con 0x5584d5c58940 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1b00a0 con 0x5584d5c52ad0 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae4ea27b80 10 client.0 _ll_register_callbacks cb 0x5584d5b4ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:32.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae41600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae40c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae42000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 0x5584d5c737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae41600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58698/0 (socket says 172.21.15.90:58698) 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae40c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49310/0 (socket says 172.21.15.90:49310) 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae42000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 0x5584d5c737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59254/0 (socket says 172.21.15.90:59254) 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.232+0000 7fae41600700 1 -- 172.21.15.90:0/3428670813 learned_addr learned my addr 172.21.15.90:0/3428670813 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 -- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 msgr2=0x5584d5c737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 0x5584d5c737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 -- 172.21.15.90:0/3428670813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 msgr2=0x5584d5c73260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 -- 172.21.15.90:0/3428670813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d5c73e10 con 0x5584d5c52ad0 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae40c00700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae41600700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae300a7860 tx=0x7fae300098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef87d04d3cdd6517 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:32.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae23400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300aaa60 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3401a920 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fae300b3160 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.233+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fae300aa5f0 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 10 client.26854 fetch_fsmap learned FSMap version 41 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5584d5cd6910 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3401a920 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8370+0+0 (secure 0 0 0) 0x7fae300a5f60 con 0x5584d5c52ad0 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 10 client.26854 fetch_fsmap finished waiting for FSMap version 41 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 20 client.26854 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5cd6f60 0x5584d5cd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 4 client.26854 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:56:32.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584d5bac570 con 0x5584d5cd6f60 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.234+0000 7fae40c00700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5cd6f60 0x5584d5cd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae40c00700 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5cd6f60 0x5584d5cd9380 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4977dd841301e61d server_cookie=506d2c489d393f8d in_seq=0 out_seq=0 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae23400700 10 client.26854 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae23400700 1 -- 172.21.15.90:0/3428670813 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584d5bac570 con 0x5584d5cd6f60 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae23400700 10 client.26854 handle_command_reply: tid=0 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 client.26854 shutdown 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 msgr2=0x5584d5c64c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fae300a7860 tx=0x7fae300098c0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 shutdown_connections 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5584d5cd6f60 0x5584d5cd9380 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5584d5c58940 0x5584d5c737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5584d5c52ad0 0x5584d5c64c20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 --2- 172.21.15.90:0/3428670813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584d5c58020 0x5584d5c73260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae23400700 0 client.26854 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:32.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 >> 172.21.15.90:0/3428670813 conn(0x5584d5b80be0 msgr2=0x5584d5c4cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 shutdown_connections 2024-06-02T15:56:32.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.235+0000 7fae4ea27b80 1 -- 172.21.15.90:0/3428670813 wait complete. 2024-06-02T15:56:32.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae4ea27b80 20 client.26854 trim_cache size 0 max 16384 2024-06-02T15:56:32.242 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:56:32.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 msgr2=0x7fae240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 0x7fae240b4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae34009ca0 tx=0x7fae34003a90 comp rx=0 tx=0).stop 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 msgr2=0x7fae3c19f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae300057c0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 shutdown_connections 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fae240b2880 0x7fae240b4d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae3c10da60 0x7fae3c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae3c104630 0x7fae3c19f870 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 --2- 172.21.15.90:0/746696075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae3c104f50 0x7fae3c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:32.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.236+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 >> 172.21.15.90:0/746696075 conn(0x7fae3c0fde60 msgr2=0x7fae3c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:32.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.237+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 shutdown_connections 2024-06-02T15:56:32.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:32.237+0000 7fae42a00700 1 -- 172.21.15.90:0/746696075 wait complete. 2024-06-02T15:56:32.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:32 smithi090 ceph-mon[31595]: pgmap v881: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:32.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:32 smithi120 ceph-mon[36451]: pgmap v881: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:32.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:32 smithi135 ceph-mon[36839]: pgmap v881: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:34.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T15:56:34.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T15:56:34.604 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:34.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:34 smithi090 ceph-mon[31595]: pgmap v882: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:34.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:34 smithi120 ceph-mon[36451]: pgmap v882: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:34.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:34 smithi135 ceph-mon[36839]: pgmap v882: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:56:36.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:36 smithi090 ceph-mon[31595]: pgmap v883: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:36.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:36 smithi120 ceph-mon[36451]: pgmap v883: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:36.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:36 smithi135 ceph-mon[36839]: pgmap v883: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:38.522 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:38 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:38.522 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:38 smithi090 ceph-mon[31595]: pgmap v884: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:38.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 -- 172.21.15.90:0/4286970084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406bbe0 msgr2=0x7f17e41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 --2- 172.21.15.90:0/4286970084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406bbe0 0x7f17e41103d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00b0a0 tx=0x7f17dc00b3b0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 -- 172.21.15.90:0/4286970084 shutdown_connections 2024-06-02T15:56:38.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 --2- 172.21.15.90:0/4286970084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406bbe0 0x7f17e41103d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 --2- 172.21.15.90:0/4286970084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e41095b0 0x7f17e4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 --2- 172.21.15.90:0/4286970084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e406b220 0x7f17e406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 -- 172.21.15.90:0/4286970084 >> 172.21.15.90:0/4286970084 conn(0x7f17e40793f0 msgr2=0x7f17e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 -- 172.21.15.90:0/4286970084 shutdown_connections 2024-06-02T15:56:38.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.595+0000 7f17eaa00700 1 -- 172.21.15.90:0/4286970084 wait complete. 2024-06-02T15:56:38.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 Processor -- start 2024-06-02T15:56:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 -- start start 2024-06-02T15:56:38.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 0x7f17e419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 0x7f17e419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 0x7f17e41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41136c0 con 0x7f17e406b220 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4113540 con 0x7f17e41095b0 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.597+0000 7f17eaa00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17e4113840 con 0x7f17e406bbe0 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 0x7f17e419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 0x7f17e419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 0x7f17e419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49336/0 (socket says 172.21.15.90:49336) 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3e00700 1 -- 172.21.15.90:0/556535693 learned_addr learned my addr 172.21.15.90:0/556535693 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e8a00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 0x7f17e41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 -- 172.21.15.90:0/556535693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 msgr2=0x7f17e41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 0x7f17e41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 -- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 msgr2=0x7f17e419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 0x7f17e419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e3400700 1 -- 172.21.15.90:0/556535693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e41a0d90 con 0x7f17e406bbe0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.598+0000 7f17e8a00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 0x7f17e41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17e3400700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 0x7f17e419c2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9f0 tx=0x7f17d000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17e0c00700 1 -- 172.21.15.90:0/556535693 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000c9c0 con 0x7f17e406bbe0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17e0c00700 1 -- 172.21.15.90:0/556535693 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d000cb60 con 0x7f17e406bbe0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e41a1080 con 0x7f17e406bbe0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17e0c00700 1 -- 172.21.15.90:0/556535693 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0018680 con 0x7f17e406bbe0 2024-06-02T15:56:38.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.599+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e41a8900 con 0x7f17e406bbe0 2024-06-02T15:56:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 Processor -- start 2024-06-02T15:56:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17e0c00700 1 -- 172.21.15.90:0/556535693 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17d0010430 con 0x7f17e406bbe0 2024-06-02T15:56:38.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 -- start start 2024-06-02T15:56:38.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f618a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f612ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f613640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555f6f570ec0 con 0x555f6f618fb0 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f6f560a60 con 0x555f6f618690 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.606+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f6f564aa0 con 0x555f6f6131c0 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f612ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f618a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f612ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49342/0 (socket says 172.21.15.90:49342) 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e3e00700 1 -- 172.21.15.90:0/2485362556 learned_addr learned my addr 172.21.15.90:0/2485362556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:38.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e3400700 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f613640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 -- 172.21.15.90:0/2485362556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 msgr2=0x555f6f613640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f613640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 -- 172.21.15.90:0/2485362556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 msgr2=0x555f6f612ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f612ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.607+0000 7f17e8a00700 1 -- 172.21.15.90:0/2485362556 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6f5294a0 con 0x555f6f618690 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17e8a00700 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f618a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d010 tx=0x7f17dc009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b55069968f4d28f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17e0c00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 0x7f17c80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17e0c00700 1 -- 172.21.15.90:0/556535693 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f17d00e26a0 con 0x7f17e406bbe0 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17e3e00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 0x7f17c80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17d9600700 1 -- 172.21.15.90:0/2485362556 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc021070 con 0x555f6f618690 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17d9600700 1 -- 172.21.15.90:0/2485362556 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc0074e0 con 0x555f6f618690 2024-06-02T15:56:38.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.608+0000 7f17d9600700 1 -- 172.21.15.90:0/2485362556 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc005280 con 0x555f6f618690 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 -- 172.21.15.90:0/2485362556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 msgr2=0x555f6f618a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17e3e00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 0x7f17c80b4dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f17d40096b0 tx=0x7f17d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f618a70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d010 tx=0x7f17dc009d90 comp rx=0 tx=0).stop 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 -- 172.21.15.90:0/2485362556 shutdown_connections 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f613640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f618a70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/2485362556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f612ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.609+0000 7f17f53b0b80 1 -- 172.21.15.90:0/2485362556 >> 172.21.15.90:0/2485362556 conn(0x555f6f540c60 msgr2=0x555f6f60d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.610+0000 7f17f53b0b80 1 -- 172.21.15.90:0/2485362556 shutdown_connections 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.610+0000 7f17f53b0b80 1 -- 172.21.15.90:0/2485362556 wait complete. 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.610+0000 7f17f53b0b80 1 -- start start 2024-06-02T15:56:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.610+0000 7f17f53b0b80 1 Processor -- start 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f61c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f61c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555f6f467020 con 0x555f6f618fb0 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x555f6f5017e0 con 0x555f6f618690 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f6f61fa00 con 0x555f6f6131c0 2024-06-02T15:56:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17f53b0b80 10 client.0 _ll_register_callbacks cb 0x555f6f50acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:58742/0 (socket says 172.21.15.90:58742) 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17e3e00700 1 -- 172.21.15.90:0/3406323044 learned_addr learned my addr 172.21.15.90:0/3406323044 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17e8a00700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f61c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.611+0000 7f17e3400700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f61c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 -- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 msgr2=0x555f6f61c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f61c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 -- 172.21.15.90:0/3406323044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 msgr2=0x555f6f61c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f61c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 -- 172.21.15.90:0/3406323044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6f4b0950 con 0x555f6f6131c0 2024-06-02T15:56:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e8a00700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f61c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17e3e00700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f17d40045f0 tx=0x7f17d40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7f886f69b13de3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.612+0000 7f17ce000700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d4011d70 con 0x555f6f6131c0 2024-06-02T15:56:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f17d401e430 con 0x555f6f6131c0 2024-06-02T15:56:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17dc0074e0 con 0x555f6f6131c0 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f17d401e630 con 0x555f6f6131c0 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17f53b0b80 10 client.26866 fetch_fsmap learned FSMap version 42 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555f6f5310e0 con 0x555f6f6131c0 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17dc0074e0 con 0x555f6f6131c0 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.613+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f17d4026070 con 0x555f6f6131c0 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:38.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 20 client.26866 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f62c610 0x555f6f62ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 4 client.26866 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f6f572e10 con 0x555f6f62c610 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.614+0000 7f17e3400700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f62c610 0x555f6f62ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.615+0000 7f17e3400700 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f62c610 0x555f6f62ea00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f99e065e0ad0c server_cookie=dc5363085bcb6c72 in_seq=0 out_seq=0 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.615+0000 7f17ce000700 10 client.26866 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.615+0000 7f17ce000700 1 -- 172.21.15.90:0/3406323044 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555f6f572e10 con 0x555f6f62c610 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.615+0000 7f17ce000700 10 client.26866 handle_command_reply: tid=0 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 client.26866 shutdown 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 msgr2=0x555f6f61ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f17d40045f0 tx=0x7f17d40047c0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 shutdown_connections 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f62c610 0x555f6f62ea00 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618fb0 0x555f6f61c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f6131c0 0x555f6f61ece0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3406323044 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618690 0x555f6f61c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17ce000700 0 client.26866 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 >> 172.21.15.90:0/3406323044 conn(0x555f6f540c60 msgr2=0x555f6f60cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 shutdown_connections 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3406323044 wait complete. 2024-06-02T15:56:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.616+0000 7f17f53b0b80 20 client.26866 trim_cache size 0 max 16384 2024-06-02T15:56:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 Processor -- start 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 -- start start 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 0x555f6f619aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 0x555f6f60d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618fb0 0x555f6f6342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41b3d50 con 0x555f6f6131c0 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41b1520 con 0x555f6f618fb0 2024-06-02T15:56:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41af370 con 0x555f6f618690 2024-06-02T15:56:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 0x555f6f619aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 0x555f6f619aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49352/0 (socket says 172.21.15.90:49352) 2024-06-02T15:56:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17e3e00700 1 -- 172.21.15.90:0/3063491984 learned_addr learned my addr 172.21.15.90:0/3063491984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.639+0000 7f17e3400700 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 0x555f6f60d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e8a00700 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618fb0 0x555f6f6342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 -- 172.21.15.90:0/3063491984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618fb0 msgr2=0x555f6f6342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618fb0 0x555f6f6342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 -- 172.21.15.90:0/3063491984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 msgr2=0x555f6f619aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 0x555f6f619aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 -- 172.21.15.90:0/3063491984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f6f61b060 con 0x555f6f618690 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.640+0000 7f17e3400700 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 0x555f6f60d570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17d00dfe40 tx=0x7f17d00dfcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103383ea8b65738e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17d8c00700 1 -- 172.21.15.90:0/3063491984 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00a50a0 con 0x555f6f618690 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17d8c00700 1 -- 172.21.15.90:0/3063491984 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d00b20b0 con 0x555f6f618690 2024-06-02T15:56:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17d8c00700 1 -- 172.21.15.90:0/3063491984 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000cd10 con 0x555f6f618690 2024-06-02T15:56:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3063491984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 msgr2=0x555f6f60d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 0x555f6f60d570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f17d00dfe40 tx=0x7f17d00dfcc0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3063491984 shutdown_connections 2024-06-02T15:56:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f618fb0 0x555f6f6342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f6131c0 0x555f6f619aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/3063491984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618690 0x555f6f60d570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.641+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3063491984 >> 172.21.15.90:0/3063491984 conn(0x555f6f540c60 msgr2=0x555f6f619390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.642+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3063491984 shutdown_connections 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.642+0000 7f17f53b0b80 1 -- 172.21.15.90:0/3063491984 wait complete. 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.642+0000 7f17f53b0b80 1 -- start start 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.642+0000 7f17f53b0b80 1 Processor -- start 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618690 0x555f6f625200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618fb0 0x555f6f621c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41af9e0 con 0x555f6f618690 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41b0e50 con 0x555f6f6131c0 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17e41b00b0 con 0x555f6f618fb0 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17f53b0b80 10 client.0 _ll_register_callbacks cb 0x555f6f50acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618690 0x555f6f625200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17e3e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:59296/0 (socket says 172.21.15.90:59296) 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17e3e00700 1 -- 172.21.15.90:0/717522302 learned_addr learned my addr 172.21.15.90:0/717522302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.643+0000 7f17e8a00700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618fb0 0x555f6f621c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 -- 172.21.15.90:0/717522302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618fb0 msgr2=0x555f6f621c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618fb0 0x555f6f621c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 -- 172.21.15.90:0/717522302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618690 msgr2=0x555f6f625200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618690 0x555f6f625200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 -- 172.21.15.90:0/717522302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f6f6221d0 con 0x555f6f6131c0 2024-06-02T15:56:38.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17e3e00700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f17d40289c0 tx=0x7f17d4009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daebba8ee89e8b62 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:38.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17ce000700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:38.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d403a020 con 0x555f6f6131c0 2024-06-02T15:56:38.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f6f56c5f0 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.644+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f17d401e7d0 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f17d401e9d0 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 10 client.26942 fetch_fsmap learned FSMap version 42 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555f6f697890 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555f6f56c5f0 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f17d4026040 con 0x555f6f6131c0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 10 client.26942 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 20 client.26942 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f697ff0 0x555f6f69a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 4 client.26942 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.645+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f6f3e32d0 con 0x555f6f697ff0 2024-06-02T15:56:38.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.646+0000 7f17e3400700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f697ff0 0x555f6f69a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.646+0000 7f17e3400700 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f697ff0 0x555f6f69a3e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a3508dd36e4ae5 server_cookie=4e5d3b775c62ae06 in_seq=0 out_seq=0 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.646+0000 7f17ce000700 10 client.26942 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.647+0000 7f17ce000700 1 -- 172.21.15.90:0/717522302 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f6f3e32d0 con 0x555f6f697ff0 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.647+0000 7f17ce000700 10 client.26942 handle_command_reply: tid=0 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.647+0000 7f17f53b0b80 1 client.26942 shutdown 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.647+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 msgr2=0x555f6f624cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.647+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f17d40289c0 tx=0x7f17d4009d20 comp rx=0 tx=0).stop 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 shutdown_connections 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x555f6f697ff0 0x555f6f69a3e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f6f618fb0 0x555f6f621c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17ce000700 0 client.26942 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x555f6f6131c0 0x555f6f624cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 --2- 172.21.15.90:0/717522302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555f6f618690 0x555f6f625200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 >> 172.21.15.90:0/717522302 conn(0x555f6f540c60 msgr2=0x555f6f619c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 shutdown_connections 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 1 -- 172.21.15.90:0/717522302 wait complete. 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.648+0000 7f17f53b0b80 20 client.26942 trim_cache size 0 max 16384 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "44d5d41b-6bb9-4cb4-95f2-4d8c86a021f5", 2024-06-02T15:56:38.647 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 msgr2=0x7f17c80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 0x7f17c80b4dc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f17d40096b0 tx=0x7f17d4009260 comp rx=0 tx=0).stop 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 msgr2=0x7f17e419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 0x7f17e419c2c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9f0 tx=0x7f17d000ed00 comp rx=0 tx=0).stop 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 shutdown_connections 2024-06-02T15:56:38.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f17c80b2900 0x7f17c80b4dc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e41095b0 0x7f17e41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17e406b220 0x7f17e419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 --2- 172.21.15.90:0/556535693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17e406bbe0 0x7f17e419c2c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 >> 172.21.15.90:0/556535693 conn(0x7f17e40793f0 msgr2=0x7f17e410e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 shutdown_connections 2024-06-02T15:56:38.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:38.649+0000 7f17eaa00700 1 -- 172.21.15.90:0/556535693 wait complete. 2024-06-02T15:56:38.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:38 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:38.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:38 smithi120 ceph-mon[36451]: pgmap v884: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:38.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:38 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:38.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:38 smithi135 ceph-mon[36839]: pgmap v884: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:39 smithi090 ceph-mon[31595]: scrub queued for path: / 2024-06-02T15:56:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:39 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:39 smithi090 ceph-mon[31595]: scrub summary: active paths [/] 2024-06-02T15:56:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:39 smithi090 ceph-mon[31595]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:39.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T15:56:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:39 smithi120 ceph-mon[36451]: scrub queued for path: / 2024-06-02T15:56:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:39 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:39 smithi120 ceph-mon[36451]: scrub summary: active paths [/] 2024-06-02T15:56:39.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:39 smithi120 ceph-mon[36451]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:39.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:39 smithi135 ceph-mon[36839]: scrub queued for path: / 2024-06-02T15:56:39.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:39 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:39.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:39 smithi135 ceph-mon[36839]: scrub summary: active paths [/] 2024-06-02T15:56:39.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:39 smithi135 ceph-mon[36839]: scrub summary: idle+waiting paths [/] 2024-06-02T15:56:40.250 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:40.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:40 smithi090 ceph-mon[31595]: pgmap v885: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8 op/s 2024-06-02T15:56:40.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:40 smithi120 ceph-mon[36451]: pgmap v885: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8 op/s 2024-06-02T15:56:40.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:40 smithi135 ceph-mon[36839]: pgmap v885: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 8 op/s 2024-06-02T15:56:41.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:41 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:56:41.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:41 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:56:41.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:41 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: scrub summary: idle 2024-06-02T15:56:42.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:42 smithi120 ceph-mon[36451]: pgmap v886: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:56:42.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: scrub summary: idle 2024-06-02T15:56:42.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:42 smithi135 ceph-mon[36839]: pgmap v886: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi120 to 176.2M 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: Unable to set osd_memory_target on smithi120 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: scrub summary: idle 2024-06-02T15:56:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:42 smithi090 ceph-mon[31595]: pgmap v886: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 10 op/s 2024-06-02T15:56:44.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:44 smithi090 ceph-mon[31595]: pgmap v887: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:44.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 -- 172.21.15.90:0/2555527297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 msgr2=0x7f665c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 --2- 172.21.15.90:0/2555527297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c10bb40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6650009990 tx=0x7f6650009ca0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 -- 172.21.15.90:0/2555527297 shutdown_connections 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 --2- 172.21.15.90:0/2555527297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c106120 0x7f665c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 --2- 172.21.15.90:0/2555527297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c10bb40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 --2- 172.21.15.90:0/2555527297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c105760 0x7f665c105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 -- 172.21.15.90:0/2555527297 >> 172.21.15.90:0/2555527297 conn(0x7f665c101250 msgr2=0x7f665c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 -- 172.21.15.90:0/2555527297 shutdown_connections 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.698+0000 7f6664000700 1 -- 172.21.15.90:0/2555527297 wait complete. 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.699+0000 7f6664000700 1 Processor -- start 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 -- start start 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 0x7f665c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1159b0 con 0x7f665c10b760 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f665c115830 con 0x7f665c105760 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6664000700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f665c115b30 con 0x7f665c106120 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:38808/0 (socket says 172.21.15.90:38808) 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6661600700 1 -- 172.21.15.90:0/679096243 learned_addr learned my addr 172.21.15.90:0/679096243 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6662000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.700+0000 7f6660c00700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 0x7f665c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 -- 172.21.15.90:0/679096243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 msgr2=0x7f665c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 -- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 msgr2=0x7f665c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 -- 172.21.15.90:0/679096243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c1a0ec0 con 0x7f665c106120 2024-06-02T15:56:44.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6661600700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6662000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6660c00700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 0x7f665c19c440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9f0 tx=0x7f664800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f665a000700 1 -- 172.21.15.90:0/679096243 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648004220 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f665a000700 1 -- 172.21.15.90:0/679096243 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66480043c0 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c1a11b0 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.701+0000 7f665a000700 1 -- 172.21.15.90:0/679096243 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664800ce60 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.702+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1a8a30 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.705+0000 7f665a000700 1 -- 172.21.15.90:0/679096243 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f66480052d0 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.707+0000 7f666e8b7b80 1 Processor -- start 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.707+0000 7f666e8b7b80 1 -- start start 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.707+0000 7f665a000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f665a000700 1 -- 172.21.15.90:0/679096243 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f66480e2770 con 0x7f665c106120 2024-06-02T15:56:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 0x5653f1b099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6662000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b04140 0x5653f1b045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653f1a61e40 con 0x5653f1b04140 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653f1a519e0 con 0x5653f1b09610 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653f1a55a20 con 0x5653f1b09f30 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 0x5653f1b099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39408/0 (socket says 172.21.15.90:39408) 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6660c00700 1 -- 172.21.15.90:0/2575437048 learned_addr learned my addr 172.21.15.90:0/2575437048 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.708+0000 7f6662000700 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b04140 0x5653f1b045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 -- 172.21.15.90:0/2575437048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 msgr2=0x5653f1b099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 0x5653f1b099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 -- 172.21.15.90:0/2575437048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b04140 msgr2=0x5653f1b045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b04140 0x5653f1b045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 -- 172.21.15.90:0/2575437048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f1a1a420 con 0x5653f1b09f30 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6661600700 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 0x5653f1b099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6662000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 0x7f66440b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fd0 tx=0x7f6654007890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f6660c00700 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66480a5a00 tx=0x7f66480b2220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b303f8fecbe59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f664ea00700 1 -- 172.21.15.90:0/2575437048 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801c040 con 0x5653f1b09f30 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f664ea00700 1 -- 172.21.15.90:0/2575437048 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66480a9980 con 0x5653f1b09f30 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.709+0000 7f664ea00700 1 -- 172.21.15.90:0/2575437048 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480a9be0 con 0x5653f1b09f30 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2575437048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 msgr2=0x5653f1b03b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66480a5a00 tx=0x7f66480b2220 comp rx=0 tx=0).stop 2024-06-02T15:56:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2575437048 shutdown_connections 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b04140 0x5653f1b045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b09610 0x5653f1b099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2575437048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09f30 0x5653f1b03b20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2575437048 >> 172.21.15.90:0/2575437048 conn(0x5653f1a31be0 msgr2=0x5653f1afe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2575437048 shutdown_connections 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.710+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2575437048 wait complete. 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 -- start start 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 Processor -- start 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b0d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653f1957fa0 con 0x5653f1b09f30 2024-06-02T15:56:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5653f19f2760 con 0x5653f1b04140 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.711+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653f1b109e0 con 0x5653f1b09610 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f666e8b7b80 10 client.0 _ll_register_callbacks cb 0x5653f19fbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39412/0 (socket says 172.21.15.90:39412) 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 -- 172.21.15.90:0/387082409 learned_addr learned my addr 172.21.15.90:0/387082409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6662000700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b0d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6661600700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 -- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 msgr2=0x5653f1b0fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 -- 172.21.15.90:0/387082409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 msgr2=0x5653f1b0d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b0d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 -- 172.21.15.90:0/387082409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f19a18d0 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.712+0000 7f6660c00700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66480a9680 tx=0x7f66480a9e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e24bd1a8387809 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664806d020 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f66480b2560 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66480a9980 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f66480c4050 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f666e8b7b80 10 client.26884 fetch_fsmap learned FSMap version 42 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5653f1a22060 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66480a9980 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.713+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f664808a080 con 0x5653f1b09610 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 10 client.26884 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 10 client.26884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:44.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 20 client.26884 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b1d3c0 0x5653f1b1f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 4 client.26884 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f1a63d90 con 0x5653f1b1d3c0 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.714+0000 7f6661600700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b1d3c0 0x5653f1b1f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.715+0000 7f6643400700 10 client.26884 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.715+0000 7f6661600700 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b1d3c0 0x5653f1b1f7b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc90b8f3d07dc544 server_cookie=c2dd85cf2f84f335 in_seq=0 out_seq=0 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.715+0000 7f6643400700 1 -- 172.21.15.90:0/387082409 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5653f1a63d90 con 0x5653f1b1d3c0 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.715+0000 7f6643400700 10 client.26884 handle_command_reply: tid=0 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 client.26884 shutdown 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 msgr2=0x5653f1b0d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66480a9680 tx=0x7f66480a9e10 comp rx=0 tx=0).stop 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 shutdown_connections 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b1d3c0 0x5653f1b1f7b0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b0d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/387082409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0d020 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f6643400700 0 client.26884 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 >> 172.21.15.90:0/387082409 conn(0x5653f1a31be0 msgr2=0x5653f1afdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 shutdown_connections 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 1 -- 172.21.15.90:0/387082409 wait complete. 2024-06-02T15:56:44.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.716+0000 7f666e8b7b80 20 client.26884 trim_cache size 0 max 16384 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 Processor -- start 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 -- start start 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1afe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b4020 con 0x5653f1b09f30 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b17f0 con 0x5653f1b04140 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.739+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1af640 con 0x5653f1b09610 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6662000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1afe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:38846/0 (socket says 172.21.15.90:38846) 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 -- 172.21.15.90:0/1744661697 learned_addr learned my addr 172.21.15.90:0/1744661697 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6660c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:39420/0 (socket says 172.21.15.90:39420) 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 -- 172.21.15.90:0/1744661697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 msgr2=0x5653f1b0aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 -- 172.21.15.90:0/1744661697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 msgr2=0x5653f1afe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1afe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 -- 172.21.15.90:0/1744661697 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f1b0b590 con 0x5653f1b04140 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.740+0000 7f6661600700 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6650000c00 tx=0x7f6650009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a091307435c60f6c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.741+0000 7f664e000700 1 -- 172.21.15.90:0/1744661697 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650005640 con 0x5653f1b04140 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.741+0000 7f664e000700 1 -- 172.21.15.90:0/1744661697 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6650003b30 con 0x5653f1b04140 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.741+0000 7f664e000700 1 -- 172.21.15.90:0/1744661697 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000faa0 con 0x5653f1b04140 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- 172.21.15.90:0/1744661697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 msgr2=0x5653f1b0bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6650000c00 tx=0x7f6650009310 comp rx=0 tx=0).stop 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- 172.21.15.90:0/1744661697 shutdown_connections 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1afe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b0bf80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/1744661697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b0aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- 172.21.15.90:0/1744661697 >> 172.21.15.90:0/1744661697 conn(0x5653f1a31be0 msgr2=0x5653f1b0a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- 172.21.15.90:0/1744661697 shutdown_connections 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- 172.21.15.90:0/1744661697 wait complete. 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.742+0000 7f666e8b7b80 1 -- start start 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 Processor -- start 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b20e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b86c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1afcb0 con 0x5653f1b09f30 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b1120 con 0x5653f1b04140 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1b0380 con 0x5653f1b09610 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f666e8b7b80 10 client.0 _ll_register_callbacks cb 0x5653f19fbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f6661600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:38862/0 (socket says 172.21.15.90:38862) 2024-06-02T15:56:44.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.743+0000 7f6661600700 1 -- 172.21.15.90:0/2420139368 learned_addr learned my addr 172.21.15.90:0/2420139368 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6660c00700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b20e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6662000700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b86c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 -- 172.21.15.90:0/2420139368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 msgr2=0x5653f1b20e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b20e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 -- 172.21.15.90:0/2420139368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 msgr2=0x5653f1b86c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b86c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 -- 172.21.15.90:0/2420139368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f1b87140 con 0x5653f1b04140 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6660c00700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b20e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6661600700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6650009960 tx=0x7f665000f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aac69972fe0d25f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6643400700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.744+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650005640 con 0x5653f1b04140 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6650003b30 con 0x5653f1b04140 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f6650003680 con 0x5653f1b04140 2024-06-02T15:56:44.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f665003e050 con 0x5653f1b04140 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f666e8b7b80 10 client.26960 fetch_fsmap learned FSMap version 42 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5653f1b882d0 con 0x5653f1b04140 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.745+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6650003b30 con 0x5653f1b04140 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 42) ==== 8366+0+0 (secure 0 0 0) 0x7f665002d030 con 0x5653f1b04140 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 10 client.26960 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 20 client.26960 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b88930 0x5653f1b8ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 4 client.26960 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653f1a5d570 con 0x5653f1b88930 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.746+0000 7f6660c00700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b88930 0x5653f1b8ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f6660c00700 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b88930 0x5653f1b8ad50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8d83403f1566eb server_cookie=817aa4e92fd936b5 in_seq=0 out_seq=0 2024-06-02T15:56:44.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f6643400700 10 client.26960 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f6643400700 1 -- 172.21.15.90:0/2420139368 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5653f1a5d570 con 0x5653f1b88930 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f6643400700 10 client.26960 handle_command_reply: tid=0 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 client.26960 shutdown 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 msgr2=0x5653f1b208e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6650009960 tx=0x7f665000f460 comp rx=0 tx=0).stop 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 shutdown_connections 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5653f1b88930 0x5653f1b8ad50 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653f1b09f30 0x5653f1b86c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5653f1b04140 0x5653f1b208e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 --2- 172.21.15.90:0/2420139368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653f1b09610 0x5653f1b20e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 >> 172.21.15.90:0/2420139368 conn(0x5653f1a31be0 msgr2=0x5653f1b0e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f6643400700 0 client.26960 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.747+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 shutdown_connections 2024-06-02T15:56:44.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f666e8b7b80 1 -- 172.21.15.90:0/2420139368 wait complete. 2024-06-02T15:56:44.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f666e8b7b80 20 client.26960 trim_cache size 0 max 16384 2024-06-02T15:56:44.752 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:56:44.752 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-06-02T15:56:44.752 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-06-02T15:56:44.752 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 msgr2=0x7f66440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 0x7f66440b4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6654009fd0 tx=0x7f6654007890 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 msgr2=0x7f665c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.748+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 0x7f665c19c440 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f664800e9f0 tx=0x7f664800ed00 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 shutdown_connections 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f66440b2880 0x7f66440b4d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f665c10b760 0x7f665c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f665c105760 0x7f665c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 --2- 172.21.15.90:0/679096243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f665c106120 0x7f665c19c440 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 >> 172.21.15.90:0/679096243 conn(0x7f665c101250 msgr2=0x7f665c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 shutdown_connections 2024-06-02T15:56:44.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:44.749+0000 7f6664000700 1 -- 172.21.15.90:0/679096243 wait complete. 2024-06-02T15:56:44.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:44 smithi120 ceph-mon[36451]: pgmap v887: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:44.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:44 smithi135 ceph-mon[36839]: pgmap v887: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-06-02T15:56:45.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:56:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:56:45.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:56:45.717 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:45 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:56:45.717 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:45 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:56:45.718 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:45 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:56:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:45 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:56:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:45 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:56:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:45 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:56:45.875 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T15:56:45.876 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:56:46.328 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:46.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:46 smithi090 ceph-mon[31595]: pgmap v888: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:46 smithi120 ceph-mon[36451]: pgmap v888: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:46.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:46 smithi135 ceph-mon[36839]: pgmap v888: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 10 op/s 2024-06-02T15:56:48.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:48 smithi090 ceph-mon[31595]: pgmap v889: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:48.623 INFO:tasks.workunit.client.0.smithi090.stderr:+ set -e 2024-06-02T15:56:48.623 INFO:tasks.workunit.client.0.smithi090.stderr:+ sudo rsync -av /tmp/multiple_rsync_payload.204082 payload.1 2024-06-02T15:56:48.666 INFO:tasks.workunit.client.0.smithi090.stdout:sending incremental file list 2024-06-02T15:56:48.670 INFO:tasks.workunit.client.0.smithi090.stderr:rsync: [Receiver] mkdir "/home/ubuntu/cephtest/mnt.0/client.0/tmp/payload.1" failed: Read-only file system (30) 2024-06-02T15:56:48.670 INFO:tasks.workunit.client.0.smithi090.stderr:rsync error: error in file IO (code 11) at main.c(783) [Receiver=3.2.3] 2024-06-02T15:56:48.694 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-06-02T15:56:48.772 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2024-06-02T15:56:48.772 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-02T15:56:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:48 smithi120 ceph-mon[36451]: pgmap v889: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:48.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:48 smithi135 ceph-mon[36839]: pgmap v889: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T15:56:50.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:50 smithi090 ceph-mon[31595]: pgmap v890: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:50.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:50 smithi120 ceph-mon[36451]: pgmap v890: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:50.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:50 smithi135 ceph-mon[36839]: pgmap v890: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-06-02T15:56:52.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:52 smithi090 ceph-mon[31595]: pgmap v891: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-06-02T15:56:52.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:52 smithi120 ceph-mon[36451]: pgmap v891: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-06-02T15:56:52.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:52 smithi135 ceph-mon[36839]: pgmap v891: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 10 op/s 2024-06-02T15:56:52.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 -- 172.21.15.90:0/2323667844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 msgr2=0x7f49b0116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:52.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 --2- 172.21.15.90:0/2323667844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b0116b00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f49a800b0a0 tx=0x7f49a800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 -- 172.21.15.90:0/2323667844 shutdown_connections 2024-06-02T15:56:52.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 --2- 172.21.15.90:0/2323667844 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b0116b00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 --2- 172.21.15.90:0/2323667844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 --2- 172.21.15.90:0/2323667844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 -- 172.21.15.90:0/2323667844 >> 172.21.15.90:0/2323667844 conn(0x7f49b007a6a0 msgr2=0x7f49b007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 -- 172.21.15.90:0/2323667844 shutdown_connections 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.842+0000 7f49b5c00700 1 -- 172.21.15.90:0/2323667844 wait complete. 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.843+0000 7f49b5c00700 1 Processor -- start 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.843+0000 7f49b5c00700 1 -- start start 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b01a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0119cf0 con 0x7f49b0075770 2024-06-02T15:56:52.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0119b70 con 0x7f49b0114710 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49b5c00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49b0119e70 con 0x7f49b0076090 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49aea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45082/0 (socket says 172.21.15.90:45082) 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49ae000700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49aea00700 1 -- 172.21.15.90:0/3544681504 learned_addr learned my addr 172.21.15.90:0/3544681504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.844+0000 7f49af400700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b01a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 -- 172.21.15.90:0/3544681504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 msgr2=0x7f49b01a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b01a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 -- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 msgr2=0x7f49b01a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 -- 172.21.15.90:0/3544681504 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b01a51b0 con 0x7f49b0076090 2024-06-02T15:56:52.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49aea00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49ae000700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01a06e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f499c00e9f0 tx=0x7f499c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004220 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b01a54a0 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c0043c0 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c005020 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.845+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b01acd20 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.846+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b006a230 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.849+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f499c004750 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.851+0000 7f49a7400700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 0x7f49940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.851+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f499c0e25c0 con 0x7f49b0076090 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.851+0000 7f49aea00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 0x7f49940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.852+0000 7f49aea00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 0x7f49940b4d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a0011550 tx=0x7f49a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:52.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:52.852+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f499c0a9f60 con 0x7f49b0076090 2024-06-02T15:56:53.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.144+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49b01a1500 con 0x7f49b0076090 2024-06-02T15:56:53.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.146+0000 7f49a7400700 1 -- 172.21.15.90:0/3544681504 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11887 (secure 0 0 0) 0x7f49b01a1500 con 0x7f49b0076090 2024-06-02T15:56:53.143 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:56:53.143 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:56:37.067686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24512,24524,24536]},"id":1}]} 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 msgr2=0x7f49940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 0x7f49940b4d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f49a0011550 tx=0x7f49a0009260 comp rx=0 tx=0).stop 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 msgr2=0x7f49b01a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01a06e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f499c00e9f0 tx=0x7f499c00ed00 comp rx=0 tx=0).stop 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 shutdown_connections 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f49940b2880 0x7f49940b4d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49b0114710 0x7f49b01a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:53.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49b0075770 0x7f49b01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 --2- 172.21.15.90:0/3544681504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49b0076090 0x7f49b01a06e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 >> 172.21.15.90:0/3544681504 conn(0x7f49b007a6a0 msgr2=0x7f49b01134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 shutdown_connections 2024-06-02T15:56:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:53.149+0000 7f49b5c00700 1 -- 172.21.15.90:0/3544681504 wait complete. 2024-06-02T15:56:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 42 2024-06-02T15:56:53.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:53 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3544681504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:53.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:53 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3544681504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:53.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:53 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3544681504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:56:54.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T15:56:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:56:54.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:54 smithi090 ceph-mon[31595]: pgmap v892: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:56:54.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:54 smithi120 ceph-mon[36451]: pgmap v892: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:56:54.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:54 smithi135 ceph-mon[36839]: pgmap v892: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s 2024-06-02T15:56:56.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:56 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:56.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:56 smithi090 ceph-mon[31595]: pgmap v893: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:56.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:56 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:56.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:56 smithi120 ceph-mon[36451]: pgmap v893: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:56.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:56 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:56.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:56 smithi135 ceph-mon[36839]: pgmap v893: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:56:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 -- 172.21.15.90:0/1287465704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 msgr2=0x7fd89c112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1287465704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c112120 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd89800b0a0 tx=0x7fd89800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 -- 172.21.15.90:0/1287465704 shutdown_connections 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1287465704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c112120 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1287465704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c075970 0x7fd89c075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1287465704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c076290 0x7fd89c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.651+0000 7fd8a3200700 1 -- 172.21.15.90:0/1287465704 >> 172.21.15.90:0/1287465704 conn(0x7fd89c0fefc0 msgr2=0x7fd89c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.652+0000 7fd8a3200700 1 -- 172.21.15.90:0/1287465704 shutdown_connections 2024-06-02T15:56:58.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.652+0000 7fd8a3200700 1 -- 172.21.15.90:0/1287465704 wait complete. 2024-06-02T15:56:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 Processor -- start 2024-06-02T15:56:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 -- start start 2024-06-02T15:56:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 0x7fd89c1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c115860 con 0x7fd89c076290 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1156e0 con 0x7fd89c10fcb0 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a3200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1159e0 con 0x7fd89c075970 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 0x7fd89c1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.653+0000 7fd8a1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 0x7fd89c1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45100/0 (socket says 172.21.15.90:45100) 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38744/0 (socket says 172.21.15.90:38744) 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1400700 1 -- 172.21.15.90:0/1710981884 learned_addr learned my addr 172.21.15.90:0/1710981884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:58.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a2800700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 -- 172.21.15.90:0/1710981884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 msgr2=0x7fd89c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 -- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 msgr2=0x7fd89c1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 0x7fd89c1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 -- 172.21.15.90:0/1710981884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c19c400 con 0x7fd89c075970 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a2800700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:56:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.654+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd89000d750 tx=0x7fd89000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.655+0000 7fd896a00700 1 -- 172.21.15.90:0/1710981884 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890016070 con 0x7fd89c075970 2024-06-02T15:56:58.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.655+0000 7fd896a00700 1 -- 172.21.15.90:0/1710981884 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd890004830 con 0x7fd89c075970 2024-06-02T15:56:58.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.655+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c19c6f0 con 0x7fd89c075970 2024-06-02T15:56:58.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:58 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:58.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:56:58 smithi090 ceph-mon[31595]: pgmap v894: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:58.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.655+0000 7fd896a00700 1 -- 172.21.15.90:0/1710981884 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8900055d0 con 0x7fd89c075970 2024-06-02T15:56:58.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.655+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c1a89d0 con 0x7fd89c075970 2024-06-02T15:56:58.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.659+0000 7fd896a00700 1 -- 172.21.15.90:0/1710981884 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd890004220 con 0x7fd89c075970 2024-06-02T15:56:58.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.660+0000 7fd8af1eab80 1 Processor -- start 2024-06-02T15:56:58.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.660+0000 7fd8af1eab80 1 -- start start 2024-06-02T15:56:58.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a2550334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255033f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a254f91e40 con 0x55a255039940 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a254f819e0 con 0x55a255039020 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a254f85a20 con 0x55a255033ad0 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8a1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a2550334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43916/0 (socket says 172.21.15.90:43916) 2024-06-02T15:56:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8a2800700 1 -- 172.21.15.90:0/3359023986 learned_addr learned my addr 172.21.15.90:0/3359023986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255033f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.661+0000 7fd896a00700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 0x7fd8840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd896a00700 1 -- 172.21.15.90:0/1710981884 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fd8900e2280 con 0x7fd89c075970 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 -- 172.21.15.90:0/3359023986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 msgr2=0x55a255033f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255033f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 -- 172.21.15.90:0/3359023986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 msgr2=0x55a2550334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a2550334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 -- 172.21.15.90:0/3359023986 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a254f4a420 con 0x55a255039020 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a2550334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.662+0000 7fd8a2800700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 0x7fd8840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.663+0000 7fd8a2800700 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd898009500 tx=0x7fd898009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548217117f69fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.663+0000 7fd88b400700 1 -- 172.21.15.90:0/3359023986 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980076e0 con 0x55a255039020 2024-06-02T15:56:58.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.663+0000 7fd88b400700 1 -- 172.21.15.90:0/3359023986 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898007d30 con 0x55a255039020 2024-06-02T15:56:58.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.663+0000 7fd88b400700 1 -- 172.21.15.90:0/3359023986 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898005730 con 0x55a255039020 2024-06-02T15:56:58.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.663+0000 7fd8a2800700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 0x7fd8840b4dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd89802a000 tx=0x7fd89802b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3359023986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 msgr2=0x55a255039400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd898009500 tx=0x7fd898009530 comp rx=0 tx=0).stop 2024-06-02T15:56:58.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3359023986 shutdown_connections 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255033f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a255039400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3359023986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a2550334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3359023986 >> 172.21.15.90:0/3359023986 conn(0x55a254f61be0 msgr2=0x55a25502e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3359023986 shutdown_connections 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.664+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3359023986 wait complete. 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 -- start start 2024-06-02T15:56:58.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 Processor -- start 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 0x55a25503c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a254e87fa0 con 0x55a255039940 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55a254f22760 con 0x55a255033ad0 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.665+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a255040460 con 0x55a255039020 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8af1eab80 10 client.0 _ll_register_callbacks cb 0x55a254f2bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 0x55a25503c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:43922/0 (socket says 172.21.15.90:43922) 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1e00700 1 -- 172.21.15.90:0/3631747342 learned_addr learned my addr 172.21.15.90:0/3631747342 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:58.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a2800700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 -- 172.21.15.90:0/3631747342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 msgr2=0x55a25503f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 -- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 msgr2=0x55a25503cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1400700 1 -- 172.21.15.90:0/3631747342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a254ed18d0 con 0x55a255039020 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.666+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 0x55a25503c960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0096d0 tx=0x7fd88c011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ce36489d663748 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd87fe00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c004830 con 0x55a255039020 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fd88c0049d0 con 0x55a255039020 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd898007d30 con 0x55a255039020 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd88c025050 con 0x55a255039020 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd8af1eab80 10 client.26908 fetch_fsmap learned FSMap version 44 2024-06-02T15:56:58.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.667+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a254f52060 con 0x55a255039020 2024-06-02T15:56:58.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898007d30 con 0x55a255039020 2024-06-02T15:56:58.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7fd88c019070 con 0x55a255039020 2024-06-02T15:56:58.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 10 client.26908 fetch_fsmap finished waiting for FSMap version 44 2024-06-02T15:56:58.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 10 client.26908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:56:58.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 20 client.26908 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a25504cfb0 0x55a25504f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 4 client.26908 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.668+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a254f93d90 con 0x55a25504cfb0 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.669+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a25504cfb0 0x55a25504f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.669+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a25504cfb0 0x55a25504f410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46a2012e51703058 server_cookie=764f18152c31b851 in_seq=0 out_seq=0 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.669+0000 7fd87fe00700 10 client.26908 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd87fe00700 1 -- 172.21.15.90:0/3631747342 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a254f93d90 con 0x55a25504cfb0 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd87fe00700 10 client.26908 handle_command_reply: tid=0 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 client.26908 shutdown 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 msgr2=0x55a25503c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 0x55a25503c960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0096d0 tx=0x7fd88c011a30 comp rx=0 tx=0).stop 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 shutdown_connections 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a25504cfb0 0x55a25504f410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a25503f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd87fe00700 0 client.26908 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3631747342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039020 0x55a25503c960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 >> 172.21.15.90:0/3631747342 conn(0x55a254f61be0 msgr2=0x55a2550324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.670+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 shutdown_connections 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.671+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3631747342 wait complete. 2024-06-02T15:56:58.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.671+0000 7fd8af1eab80 20 client.26908 trim_cache size 0 max 16384 2024-06-02T15:56:58.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.687+0000 7fd8af1eab80 1 Processor -- start 2024-06-02T15:56:58.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.687+0000 7fd8af1eab80 1 -- start start 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255036b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a2550370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1b3d60 con 0x55a255039940 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1b1560 con 0x55a255039020 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1af3b0 con 0x55a255033ad0 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45130/0 (socket says 172.21.15.90:45130) 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 -- 172.21.15.90:0/844348431 learned_addr learned my addr 172.21.15.90:0/844348431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255036b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a1400700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a2550370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 -- 172.21.15.90:0/844348431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 msgr2=0x55a255036b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255036b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 -- 172.21.15.90:0/844348431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 msgr2=0x55a2550370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.688+0000 7fd8a2800700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a2550370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd8a2800700 1 -- 172.21.15.90:0/844348431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a255038aa0 con 0x55a255039940 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255036b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd8a1400700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a2550370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd8a2800700 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd898004fa0 tx=0x7fd89800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3314447e24a295 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd88aa00700 1 -- 172.21.15.90:0/844348431 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898005a00 con 0x55a255039940 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd88aa00700 1 -- 172.21.15.90:0/844348431 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898005040 con 0x55a255039940 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.689+0000 7fd88aa00700 1 -- 172.21.15.90:0/844348431 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898007c10 con 0x55a255039940 2024-06-02T15:56:58.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 -- 172.21.15.90:0/844348431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 msgr2=0x55a25503c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd898004fa0 tx=0x7fd89800b3b0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 -- 172.21.15.90:0/844348431 shutdown_connections 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039940 0x55a25503c920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255033ad0 0x55a255036b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/844348431 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255039020 0x55a2550370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 -- 172.21.15.90:0/844348431 >> 172.21.15.90:0/844348431 conn(0x55a254f61be0 msgr2=0x55a2550324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 -- 172.21.15.90:0/844348431 shutdown_connections 2024-06-02T15:56:58.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.690+0000 7fd8af1eab80 1 -- 172.21.15.90:0/844348431 wait complete. 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.691+0000 7fd8af1eab80 1 -- start start 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.691+0000 7fd8af1eab80 1 Processor -- start 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a255045c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039020 0x55a2550543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1afa20 con 0x55a255039020 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1b0e90 con 0x55a255033ad0 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c1b00f0 con 0x55a255039940 2024-06-02T15:56:58.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8af1eab80 10 client.0 _ll_register_callbacks cb 0x55a254f2bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039020 0x55a2550543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a2800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:38786/0 (socket says 172.21.15.90:38786) 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a2800700 1 -- 172.21.15.90:0/3114233301 learned_addr learned my addr 172.21.15.90:0/3114233301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a255045c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 -- 172.21.15.90:0/3114233301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 msgr2=0x55a255054900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 -- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039020 msgr2=0x55a2550543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039020 0x55a2550543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a1e00700 1 -- 172.21.15.90:0/3114233301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a255054ef0 con 0x55a255033ad0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.692+0000 7fd8a2800700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd8a1e00700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a255045c60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8900a9420 tx=0x7fd8900a9380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=861bb0fc6f94b398 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:56:58.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd87fe00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890004510 con 0x55a255033ad0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fd8900e5db0 con 0x55a255033ad0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd898005040 con 0x55a255033ad0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd8900a8fa0 con 0x55a255033ad0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd8af1eab80 10 client.26984 fetch_fsmap learned FSMap version 44 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a2550c02b0 con 0x55a255033ad0 2024-06-02T15:56:58.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.693+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898005040 con 0x55a255033ad0 2024-06-02T15:56:58.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7fd8900b1e60 con 0x55a255033ad0 2024-06-02T15:56:58.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 10 client.26984 fetch_fsmap finished waiting for FSMap version 44 2024-06-02T15:56:58.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 10 client.26984 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.g 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 20 client.26984 populate_metadata read hostname 'smithi090' 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a2550c0960 0x55a2550c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 4 client.26984 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 --> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a254f8d570 con 0x55a2550c0960 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.694+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a2550c0960 0x55a2550c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8a1400700 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a2550c0960 0x55a2550c2d80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a6b980d71eaa238 server_cookie=5a035cdebd805864 in_seq=0 out_seq=0 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd87fe00700 10 client.26984 ms_handle_connect on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd87fe00700 1 -- 172.21.15.90:0/3114233301 <== mds.2 v2:172.21.15.90:6838/1414255295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a254f8d570 con 0x55a2550c0960 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd87fe00700 10 client.26984 handle_command_reply: tid=0 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 client.26984 shutdown 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 msgr2=0x55a255045c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a255045c60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8900a9420 tx=0x7fd8900a9380 comp rx=0 tx=0).stop 2024-06-02T15:56:58.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 shutdown_connections 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:6838/1414255295,v1:172.21.15.90:6839/1414255295] conn(0x55a2550c0960 0x55a2550c2d80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a255039940 0x55a255054900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55a255033ad0 0x55a255045c60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd87fe00700 0 client.26984 ms_handle_reset on v2:172.21.15.90:6838/1414255295 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 --2- 172.21.15.90:0/3114233301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a255039020 0x55a2550543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.695+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 >> 172.21.15.90:0/3114233301 conn(0x55a254f61be0 msgr2=0x55a25503c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.696+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 shutdown_connections 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.696+0000 7fd8af1eab80 1 -- 172.21.15.90:0/3114233301 wait complete. 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.696+0000 7fd8af1eab80 20 client.26984 trim_cache size 0 max 16384 2024-06-02T15:56:58.695 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:56:58.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 msgr2=0x7fd8840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 0x7fd8840b4dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd89802a000 tx=0x7fd89802b000 comp rx=0 tx=0).stop 2024-06-02T15:56:58.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 msgr2=0x7fd89c1a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:56:58.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd89000d750 tx=0x7fd89000da60 comp rx=0 tx=0).stop 2024-06-02T15:56:58.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 shutdown_connections 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fd8840b2900 0x7fd8840b4dc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd89c10fcb0 0x7fd89c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd89c075970 0x7fd89c1a1a50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 --2- 172.21.15.90:0/1710981884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd89c076290 0x7fd89c1a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 >> 172.21.15.90:0/1710981884 conn(0x7fd89c0fefc0 msgr2=0x7fd89c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 shutdown_connections 2024-06-02T15:56:58.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:56:58.697+0000 7fd8a3200700 1 -- 172.21.15.90:0/1710981884 wait complete. 2024-06-02T15:56:58.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:58 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:58.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:56:58 smithi120 ceph-mon[36451]: pgmap v894: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:58.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:58 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:56:58.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:56:58 smithi135 ceph-mon[36839]: pgmap v894: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:56:59.892 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T15:57:00.047 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi090 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-06-02T15:57:00.280 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=afac631631f6485d95c88ad686be3bf4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi090 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-06-02T15:57:00.283 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-02T15:57:00.293 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:57:00.309 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-02T15:57:00.319 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-02T15:57:00.337 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:00.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:00 smithi090 ceph-mon[31595]: pgmap v895: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:00.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:00 smithi120 ceph-mon[36451]: pgmap v895: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:00.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:00 smithi135 ceph-mon[36839]: pgmap v895: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:02.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:02 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:02.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:02 smithi090 ceph-mon[31595]: pgmap v896: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:02.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:02 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:02.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:02 smithi120 ceph-mon[36451]: pgmap v896: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:02 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:02 smithi135 ceph-mon[36839]: pgmap v896: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:03.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 -- 172.21.15.90:0/813623724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98076200 msgr2=0x7f1e98113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 --2- 172.21.15.90:0/813623724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98076200 0x7f1e98113f30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e840098f0 tx=0x7f1e84009c00 comp rx=0 tx=0).stop 2024-06-02T15:57:03.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 -- 172.21.15.90:0/813623724 shutdown_connections 2024-06-02T15:57:03.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 --2- 172.21.15.90:0/813623724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98114470 0x7f1e98116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 --2- 172.21.15.90:0/813623724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 0x7f1e98075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 --2- 172.21.15.90:0/813623724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98076200 0x7f1e98113f30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.208+0000 7f1e97200700 1 -- 172.21.15.90:0/813623724 >> 172.21.15.90:0/813623724 conn(0x7f1e980fef90 msgr2=0x7f1e981013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.209+0000 7f1e97200700 1 -- 172.21.15.90:0/813623724 shutdown_connections 2024-06-02T15:57:03.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.209+0000 7f1e97200700 1 -- 172.21.15.90:0/813623724 wait complete. 2024-06-02T15:57:03.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 Processor -- start 2024-06-02T15:57:03.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 -- start start 2024-06-02T15:57:03.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 0x7f1e981a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 0x7f1e981a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 0x7f1e981a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e98119ca0 con 0x7f1e98076200 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e98119b20 con 0x7f1e98114470 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.210+0000 7f1e97200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e98119e20 con 0x7f1e980758e0 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 0x7f1e981a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e95e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 0x7f1e981a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 0x7f1e981a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53428/0 (socket says 172.21.15.90:53428) 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 0x7f1e981a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e95400700 1 -- 172.21.15.90:0/3289682570 learned_addr learned my addr 172.21.15.90:0/3289682570 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 -- 172.21.15.90:0/3289682570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 msgr2=0x7f1e981a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 0x7f1e981a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 -- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 msgr2=0x7f1e981a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 0x7f1e981a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 -- 172.21.15.90:0/3289682570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e981a5240 con 0x7f1e98114470 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.211+0000 7f1e96800700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 0x7f1e981a4b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009cd0 tx=0x7f1e9000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.212+0000 7f1e8ea00700 1 -- 172.21.15.90:0/3289682570 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000cce0 con 0x7f1e98114470 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.212+0000 7f1e8ea00700 1 -- 172.21.15.90:0/3289682570 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e90003a00 con 0x7f1e98114470 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.212+0000 7f1e8ea00700 1 -- 172.21.15.90:0/3289682570 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e900044f0 con 0x7f1e98114470 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.212+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e981a54d0 con 0x7f1e98114470 2024-06-02T15:57:03.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.212+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e981acdb0 con 0x7f1e98114470 2024-06-02T15:57:03.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.215+0000 7f1e8ea00700 1 -- 172.21.15.90:0/3289682570 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e9001a020 con 0x7f1e98114470 2024-06-02T15:57:03.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 Processor -- start 2024-06-02T15:57:03.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 -- start start 2024-06-02T15:57:03.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1e8ea00700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 0x7f1e7c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1e8ea00700 1 -- 172.21.15.90:0/3289682570 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e900e1dd0 con 0x7f1e98114470 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacbcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55abaac1ae40 con 0x55abaacbd390 2024-06-02T15:57:03.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1e95400700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 0x7f1e7c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abaac0a9e0 con 0x55abaacc27c0 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.217+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55abaac0ea20 con 0x55abaacc30e0 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e96800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e96800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53438/0 (socket says 172.21.15.90:53438) 2024-06-02T15:57:03.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e96800700 1 -- 172.21.15.90:0/2697302460 learned_addr learned my addr 172.21.15.90:0/2697302460 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95400700 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacbcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 -- 172.21.15.90:0/2697302460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 msgr2=0x55abaacbcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacbcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 -- 172.21.15.90:0/2697302460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 msgr2=0x55abaacbd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95e00700 1 -- 172.21.15.90:0/2697302460 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abaabd3420 con 0x55abaacc27c0 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e96800700 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.218+0000 7f1e95400700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 0x7f1e7c0b4dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009680 tx=0x7f1e8400fd00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.219+0000 7f1e95e00700 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc2ba0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009cb0 tx=0x7f1e88011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1636b949eca2c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.219+0000 7f1e83400700 1 -- 172.21.15.90:0/2697302460 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011d20 con 0x55abaacc27c0 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.219+0000 7f1e83400700 1 -- 172.21.15.90:0/2697302460 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88004500 con 0x55abaacc27c0 2024-06-02T15:57:03.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.219+0000 7f1e83400700 1 -- 172.21.15.90:0/2697302460 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88005310 con 0x55abaacc27c0 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 -- 172.21.15.90:0/2697302460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 msgr2=0x55abaacc2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc2ba0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009cb0 tx=0x7f1e88011440 comp rx=0 tx=0).stop 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 -- 172.21.15.90:0/2697302460 shutdown_connections 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc2ba0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/2697302460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacbcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 -- 172.21.15.90:0/2697302460 >> 172.21.15.90:0/2697302460 conn(0x55abaabeabe0 msgr2=0x55abaacb7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 -- 172.21.15.90:0/2697302460 shutdown_connections 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.220+0000 7f1ea759cb80 1 -- 172.21.15.90:0/2697302460 wait complete. 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.221+0000 7f1ea759cb80 1 -- start start 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.221+0000 7f1ea759cb80 1 Processor -- start 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaacc8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaacc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacc6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55abaab10fa0 con 0x55abaacc27c0 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55abaabab760 con 0x55abaacc30e0 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55abaacc9c00 con 0x55abaacbd390 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1ea759cb80 10 client.0 _ll_register_callbacks cb 0x55abaabb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaacc61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaacc61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53448/0 (socket says 172.21.15.90:53448) 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95400700 1 -- 172.21.15.90:0/549907368 learned_addr learned my addr 172.21.15.90:0/549907368 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95e00700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaacc8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e96800700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacc6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95e00700 1 -- 172.21.15.90:0/549907368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 msgr2=0x55abaacc6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95e00700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacc6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95e00700 1 -- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 msgr2=0x55abaacc61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.222+0000 7f1e95e00700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaacc61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e95e00700 1 -- 172.21.15.90:0/549907368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abaab5a8d0 con 0x55abaacbd390 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e95e00700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaacc8e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73583c84fe7b4c01 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e77e00700 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004960 con 0x55abaacbd390 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88004500 con 0x55abaacbd390 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e88004f90 con 0x55abaacbd390 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1e88030050 con 0x55abaacbd390 2024-06-02T15:57:03.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.223+0000 7f1ea759cb80 10 client.26926 fetch_fsmap learned FSMap version 45 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55abaabdb060 con 0x55abaacbd390 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88004500 con 0x55abaacbd390 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f1e88018070 con 0x55abaacbd390 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 10 client.26926 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 10 client.26926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 20 client.26926 populate_metadata read hostname 'smithi090' 2024-06-02T15:57:03.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaacd65e0 0x55abaacd89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 4 client.26926 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.224+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abaac1cd90 con 0x55abaacd65e0 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.225+0000 7f1e95400700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaacd65e0 0x55abaacd89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.225+0000 7f1e95400700 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaacd65e0 0x55abaacd89d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8537378ec3664ee1 server_cookie=d7fe6f45228028a3 in_seq=0 out_seq=0 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.225+0000 7f1e77e00700 10 client.26926 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1e77e00700 1 -- 172.21.15.90:0/549907368 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55abaac1cd90 con 0x55abaacd65e0 2024-06-02T15:57:03.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1e77e00700 10 client.26926 handle_command_reply: tid=0 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 client.26926 shutdown 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 msgr2=0x55abaacc8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaacc8e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1e88000c00 tx=0x7f1e88005d90 comp rx=0 tx=0).stop 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 shutdown_connections 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaacd65e0 0x55abaacd89d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacc6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaacc8e30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/549907368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaacc61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1e77e00700 0 client.26926 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 >> 172.21.15.90:0/549907368 conn(0x55abaabeabe0 msgr2=0x55abaacb6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.226+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 shutdown_connections 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.227+0000 7f1ea759cb80 1 -- 172.21.15.90:0/549907368 wait complete. 2024-06-02T15:57:03.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.227+0000 7f1ea759cb80 20 client.26926 trim_cache size 0 max 16384 2024-06-02T15:57:03.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 Processor -- start 2024-06-02T15:57:03.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 -- start start 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacb7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b8200 con 0x55abaacbd390 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b5a60 con 0x55abaacc27c0 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.243+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b38b0 con 0x55abaacc30e0 2024-06-02T15:57:03.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53464/0 (socket says 172.21.15.90:53464) 2024-06-02T15:57:03.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 -- 172.21.15.90:0/3047299665 learned_addr learned my addr 172.21.15.90:0/3047299665 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:03.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95400700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e96800700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacb7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 -- 172.21.15.90:0/3047299665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 msgr2=0x55abaacb7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacb7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 -- 172.21.15.90:0/3047299665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 msgr2=0x55abaacc3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95e00700 1 -- 172.21.15.90:0/3047299665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abaacc4740 con 0x55abaacbd390 2024-06-02T15:57:03.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.244+0000 7f1e95400700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.245+0000 7f1e95e00700 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e88003c60 tx=0x7f1e88011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842fff75c59b4b99 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.245+0000 7f1e82a00700 1 -- 172.21.15.90:0/3047299665 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88022400 con 0x55abaacbd390 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.245+0000 7f1e82a00700 1 -- 172.21.15.90:0/3047299665 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88027070 con 0x55abaacbd390 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.245+0000 7f1e82a00700 1 -- 172.21.15.90:0/3047299665 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e880038e0 con 0x55abaacbd390 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 -- 172.21.15.90:0/3047299665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 msgr2=0x55abaacbcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e88003c60 tx=0x7f1e88011440 comp rx=0 tx=0).stop 2024-06-02T15:57:03.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 -- 172.21.15.90:0/3047299665 shutdown_connections 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacc30e0 0x55abaacb7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacbd390 0x55abaacbcd70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/3047299665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc27c0 0x55abaacc3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 -- 172.21.15.90:0/3047299665 >> 172.21.15.90:0/3047299665 conn(0x55abaabeabe0 msgr2=0x55abaacc6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.246+0000 7f1ea759cb80 1 -- 172.21.15.90:0/3047299665 shutdown_connections 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.247+0000 7f1ea759cb80 1 -- 172.21.15.90:0/3047299665 wait complete. 2024-06-02T15:57:03.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.247+0000 7f1ea759cb80 1 -- start start 2024-06-02T15:57:03.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.247+0000 7f1ea759cb80 1 Processor -- start 2024-06-02T15:57:03.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaaccf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacdd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b3f20 con 0x55abaacc27c0 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b5390 con 0x55abaacc30e0 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e981b45f0 con 0x55abaacbd390 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1ea759cb80 10 client.0 _ll_register_callbacks cb 0x55abaabb4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:57:03.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1e96800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacdd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53480/0 (socket says 172.21.15.90:53480) 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1e95400700 1 -- 172.21.15.90:0/1424876633 learned_addr learned my addr 172.21.15.90:0/1424876633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.248+0000 7f1e95e00700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaaccf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 -- 172.21.15.90:0/1424876633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 msgr2=0x55abaaccf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaaccf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 -- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 msgr2=0x55abaaccf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:03.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 -- 172.21.15.90:0/1424876633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abaacde190 con 0x55abaacc30e0 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e95e00700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaaccf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e95400700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e96800700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacdd980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1e900a5060 tx=0x7f1e90004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c612bf92597671dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e77e00700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e900a8af0 con 0x55abaacc30e0 2024-06-02T15:57:03.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1e90004bd0 con 0x55abaacc30e0 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e88027070 con 0x55abaacc30e0 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.249+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1e900a96e0 con 0x55abaacc30e0 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 10 client.27002 fetch_fsmap learned FSMap version 45 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55abaad40d80 con 0x55abaacc30e0 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e88027070 con 0x55abaacc30e0 2024-06-02T15:57:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7f1e900b19b0 con 0x55abaacc30e0 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 10 client.27002 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 10 client.27002 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 20 client.27002 populate_metadata read hostname 'smithi090' 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaad41560 0x55abaad43980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 4 client.27002 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 --> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abaac16570 con 0x55abaad41560 2024-06-02T15:57:03.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.250+0000 7f1e95e00700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaad41560 0x55abaad43980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1e95e00700 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaad41560 0x55abaad43980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ad4349918428084 server_cookie=2ae8a90d75c375b7 in_seq=0 out_seq=0 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1e77e00700 10 client.27002 ms_handle_connect on v2:172.21.15.90:6840/1722073812 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1e77e00700 1 -- 172.21.15.90:0/1424876633 <== mds.1 v2:172.21.15.90:6840/1722073812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abaac16570 con 0x55abaad41560 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1e77e00700 10 client.27002 handle_command_reply: tid=0 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1ea759cb80 1 client.27002 shutdown 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 msgr2=0x55abaacdd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacdd980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1e900a5060 tx=0x7f1e90004970 comp rx=0 tx=0).stop 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 shutdown_connections 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.251+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:6840/1722073812,v1:172.21.15.90:6841/1722073812] conn(0x55abaad41560 0x55abaad43980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55abaacc30e0 0x55abaacdd980 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abaacbd390 0x55abaaccf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 --2- 172.21.15.90:0/1424876633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abaacc27c0 0x55abaaccf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1e77e00700 0 client.27002 ms_handle_reset on v2:172.21.15.90:6840/1722073812 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 >> 172.21.15.90:0/1424876633 conn(0x55abaabeabe0 msgr2=0x55abaacc4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 shutdown_connections 2024-06-02T15:57:03.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 1 -- 172.21.15.90:0/1424876633 wait complete. 2024-06-02T15:57:03.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.252+0000 7f1ea759cb80 20 client.27002 trim_cache size 0 max 16384 2024-06-02T15:57:03.258 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:57:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 msgr2=0x7f1e7c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 0x7f1e7c0b4dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009680 tx=0x7f1e8400fd00 comp rx=0 tx=0).stop 2024-06-02T15:57:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 msgr2=0x7f1e981a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 0x7f1e981a4b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009cd0 tx=0x7f1e9000c620 comp rx=0 tx=0).stop 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 shutdown_connections 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f1e7c0b2900 0x7f1e7c0b4dc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e98114470 0x7f1e981a4b00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e980758e0 0x7f1e981a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 --2- 172.21.15.90:0/3289682570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e98076200 0x7f1e981a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 >> 172.21.15.90:0/3289682570 conn(0x7f1e980fef90 msgr2=0x7f1e98100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:03.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 shutdown_connections 2024-06-02T15:57:03.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:03.253+0000 7f1e97200700 1 -- 172.21.15.90:0/3289682570 wait complete. 2024-06-02T15:57:04.108 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T15:57:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:04.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:04 smithi090 ceph-mon[31595]: pgmap v897: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:04.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:04 smithi120 ceph-mon[36451]: pgmap v897: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:04.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:04 smithi135 ceph-mon[36839]: pgmap v897: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:06.571 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:06 smithi090 ceph-mon[31595]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:06.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:06 smithi120 ceph-mon[36451]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:06.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:06 smithi135 ceph-mon[36839]: pgmap v898: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.910+0000 7fb43e600700 1 -- 172.21.15.90:0/3671649906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb438075770 msgr2=0x7fb438075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.910+0000 7fb43e600700 1 --2- 172.21.15.90:0/3671649906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb438075770 0x7fb438075b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb4280098f0 tx=0x7fb428009c00 comp rx=0 tx=0).stop 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 -- 172.21.15.90:0/3671649906 shutdown_connections 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 --2- 172.21.15.90:0/3671649906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb43810ffa0 0x7fb438112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 --2- 172.21.15.90:0/3671649906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb438075770 0x7fb438075b50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 --2- 172.21.15.90:0/3671649906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 -- 172.21.15.90:0/3671649906 >> 172.21.15.90:0/3671649906 conn(0x7fb43807a6a0 msgr2=0x7fb43807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 -- 172.21.15.90:0/3671649906 shutdown_connections 2024-06-02T15:57:06.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.911+0000 7fb43e600700 1 -- 172.21.15.90:0/3671649906 wait complete. 2024-06-02T15:57:06.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.912+0000 7fb43e600700 1 Processor -- start 2024-06-02T15:57:06.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.912+0000 7fb43e600700 1 -- start start 2024-06-02T15:57:06.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.912+0000 7fb43e600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 0x7fb43819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.912+0000 7fb43e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb43e600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 0x7fb4381a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb43e600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb438115930 con 0x7fb438076090 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb43e600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381157b0 con 0x7fb43810ffa0 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb43e600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb438115ab0 con 0x7fb438075770 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb436a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb436a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53486/0 (socket says 172.21.15.90:53486) 2024-06-02T15:57:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb436a00700 1 -- 172.21.15.90:0/2455066864 learned_addr learned my addr 172.21.15.90:0/2455066864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 0x7fb43819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437e00700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 0x7fb4381a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 -- 172.21.15.90:0/2455066864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 msgr2=0x7fb4381a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 0x7fb4381a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 -- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 msgr2=0x7fb43819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437400700 1 -- 172.21.15.90:0/2455066864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4381a0e50 con 0x7fb438075770 2024-06-02T15:57:06.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb436a00700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:06.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.913+0000 7fb437e00700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 0x7fb4381a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:06.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 0x7fb43819be40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb42800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb42fe00700 1 -- 172.21.15.90:0/2455066864 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428005040 con 0x7fb438075770 2024-06-02T15:57:06.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb42fe00700 1 -- 172.21.15.90:0/2455066864 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb428005690 con 0x7fb438075770 2024-06-02T15:57:06.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4381a10e0 con 0x7fb438075770 2024-06-02T15:57:06.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb42fe00700 1 -- 172.21.15.90:0/2455066864 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280038e0 con 0x7fb438075770 2024-06-02T15:57:06.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.914+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4381a89c0 con 0x7fb438075770 2024-06-02T15:57:06.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.920+0000 7fb42fe00700 1 -- 172.21.15.90:0/2455066864 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb42800b760 con 0x7fb438075770 2024-06-02T15:57:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.920+0000 7fb448fadb80 1 Processor -- start 2024-06-02T15:57:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.920+0000 7fb448fadb80 1 -- start start 2024-06-02T15:57:06.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564775735e40 con 0x5647757dceb0 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647757259e0 con 0x5647757d7880 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb448fadb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564775729a20 con 0x5647757dd7d0 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb437400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53502/0 (socket says 172.21.15.90:53502) 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb437e00700 1 -- 172.21.15.90:0/968970759 learned_addr learned my addr 172.21.15.90:0/968970759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.921+0000 7fb436a00700 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 -- 172.21.15.90:0/968970759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 msgr2=0x5647757d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 -- 172.21.15.90:0/968970759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 msgr2=0x5647757d7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 -- 172.21.15.90:0/968970759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647756ee420 con 0x5647757dceb0 2024-06-02T15:57:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb42fe00700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 0x7fb41c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb42fe00700 1 -- 172.21.15.90:0/2455066864 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb4280e3310 con 0x7fb438075770 2024-06-02T15:57:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437e00700 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb430003da0 tx=0x7fb430009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2947cf98adfdb939 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 0x7fb41c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.922+0000 7fb42cc00700 1 -- 172.21.15.90:0/968970759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430019020 con 0x5647757dceb0 2024-06-02T15:57:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb42cc00700 1 -- 172.21.15.90:0/968970759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4300092d0 con 0x5647757dceb0 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb42cc00700 1 -- 172.21.15.90:0/968970759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430007720 con 0x5647757dceb0 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 -- 172.21.15.90:0/968970759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 msgr2=0x5647757dd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb437400700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 0x7fb41c0b4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb4280aa310 tx=0x7fb4280e6df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb430003da0 tx=0x7fb430009920 comp rx=0 tx=0).stop 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 -- 172.21.15.90:0/968970759 shutdown_connections 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757dd290 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 --2- 172.21.15.90:0/968970759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.923+0000 7fb448fadb80 1 -- 172.21.15.90:0/968970759 >> 172.21.15.90:0/968970759 conn(0x564775705be0 msgr2=0x5647757d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.924+0000 7fb448fadb80 1 -- 172.21.15.90:0/968970759 shutdown_connections 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.924+0000 7fb448fadb80 1 -- 172.21.15.90:0/968970759 wait complete. 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.924+0000 7fb448fadb80 1 -- start start 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.924+0000 7fb448fadb80 1 Processor -- start 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56477562bfa0 con 0x5647757dceb0 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5647756c6760 con 0x5647757d7880 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647757e4240 con 0x5647757dd7d0 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb448fadb80 10 client.0 _ll_register_callbacks cb 0x5647756cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb437e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb437e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47346/0 (socket says 172.21.15.90:47346) 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb437e00700 1 -- 172.21.15.90:0/2186415304 learned_addr learned my addr 172.21.15.90:0/2186415304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb436a00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.925+0000 7fb437e00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 -- 172.21.15.90:0/2186415304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 msgr2=0x5647757e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 -- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 msgr2=0x5647757e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 -- 172.21.15.90:0/2186415304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647756758d0 con 0x5647757dceb0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb437e00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb436a00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757e0840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb438111c80 tx=0x7fb424009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5490ad41fa7c386b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb421600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424004050 con 0x5647757dceb0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.926+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb4240041f0 con 0x5647757dceb0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4300092d0 con 0x5647757dceb0 2024-06-02T15:57:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb4240043f0 con 0x5647757dceb0 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb448fadb80 10 client.16713 fetch_fsmap learned FSMap version 45 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5647756f6060 con 0x5647757dceb0 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4300092d0 con 0x5647757dceb0 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fb424003a70 con 0x5647757dceb0 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb448fadb80 10 client.16713 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.927+0000 7fb448fadb80 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb448fadb80 20 client.16713 populate_metadata read hostname 'smithi090' 2024-06-02T15:57:06.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5647757f0d90 0x5647757f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb448fadb80 4 client.16713 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564775737d90 con 0x5647757f0d90 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb437e00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5647757f0d90 0x5647757f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb437e00700 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5647757f0d90 0x5647757f3180 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a7b0b42fe83c47 server_cookie=929d0ca03ede29d4 in_seq=0 out_seq=0 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.928+0000 7fb421600700 10 client.16713 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb421600700 1 -- 172.21.15.90:0/2186415304 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564775737d90 con 0x5647757f0d90 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb421600700 10 client.16713 handle_command_reply: tid=0 2024-06-02T15:57:06.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 client.16713 shutdown 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 msgr2=0x5647757e0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757e0840 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb438111c80 tx=0x7fb424009630 comp rx=0 tx=0).stop 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 shutdown_connections 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x5647757f0d90 0x5647757f3180 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dd7d0 0x5647757e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 --2- 172.21.15.90:0/2186415304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dceb0 0x5647757e0840 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 >> 172.21.15.90:0/2186415304 conn(0x564775705be0 msgr2=0x5647757d6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb421600700 0 client.16713 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:57:06.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 shutdown_connections 2024-06-02T15:57:06.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 1 -- 172.21.15.90:0/2186415304 wait complete. 2024-06-02T15:57:06.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.930+0000 7fb448fadb80 20 client.16713 trim_cache size 0 max 16384 2024-06-02T15:57:06.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 Processor -- start 2024-06-02T15:57:06.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 -- start start 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757ed820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b3db0 con 0x5647757dd7d0 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b1610 con 0x5647757d7880 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb448fadb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381af460 con 0x5647757dceb0 2024-06-02T15:57:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.947+0000 7fb436a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757ed820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:59498/0 (socket says 172.21.15.90:59498) 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 -- 172.21.15.90:0/1401441414 learned_addr learned my addr 172.21.15.90:0/1401441414 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb437e00700 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 -- 172.21.15.90:0/1401441414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 msgr2=0x5647757d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 -- 172.21.15.90:0/1401441414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 msgr2=0x5647757ed820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757ed820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 -- 172.21.15.90:0/1401441414 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647757cc4c0 con 0x5647757dceb0 2024-06-02T15:57:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb437e00700 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:57:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb436a00700 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb424005d10 tx=0x7fb424009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16af7258d63af1a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.948+0000 7fb423e00700 1 -- 172.21.15.90:0/1401441414 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424003fe0 con 0x5647757dceb0 2024-06-02T15:57:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb423e00700 1 -- 172.21.15.90:0/1401441414 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb424004180 con 0x5647757dceb0 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb423e00700 1 -- 172.21.15.90:0/1401441414 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424005070 con 0x5647757dceb0 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 -- 172.21.15.90:0/1401441414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 msgr2=0x5647757ed2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb424005d10 tx=0x7fb424009ab0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 -- 172.21.15.90:0/1401441414 shutdown_connections 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757ed820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1401441414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757ed2e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 -- 172.21.15.90:0/1401441414 >> 172.21.15.90:0/1401441414 conn(0x564775705be0 msgr2=0x5647757da9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 -- 172.21.15.90:0/1401441414 shutdown_connections 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.949+0000 7fb448fadb80 1 -- 172.21.15.90:0/1401441414 wait complete. 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.950+0000 7fb448fadb80 1 -- start start 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.950+0000 7fb448fadb80 1 Processor -- start 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381afad0 con 0x5647757dd7d0 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b0f40 con 0x5647757d7880 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b01a0 con 0x5647757dceb0 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb448fadb80 10 client.0 _ll_register_callbacks cb 0x5647756cfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb436a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47358/0 (socket says 172.21.15.90:47358) 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 -- 172.21.15.90:0/1876247225 learned_addr learned my addr 172.21.15.90:0/1876247225 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:06.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 -- 172.21.15.90:0/1876247225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 msgr2=0x5647757e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 -- 172.21.15.90:0/1876247225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 msgr2=0x5647757f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.951+0000 7fb437e00700 1 -- 172.21.15.90:0/1876247225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647757f7f80 con 0x5647757d7880 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb437e00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e10 tx=0x7fb430009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c87f97af3f8bf88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb421600700 10 client.0 ms_handle_connect on v2:172.21.15.120:3300/0 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430019040 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb424004180 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mon.2 v2:172.21.15.120:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb43002fe00 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb43000ec60 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb448fadb80 10 client.27020 fetch_fsmap learned FSMap version 45 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.952+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564775863910 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mon.2 v2:172.21.15.120:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb424004180 con 0x5647757d7880 2024-06-02T15:57:06.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mon.2 v2:172.21.15.120:3300/0 5 ==== fsmap(e 45) ==== 8350+0+0 (secure 0 0 0) 0x7fb430007620 con 0x5647757d7880 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 10 client.27020 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 20 client.27020 populate_metadata read hostname 'smithi090' 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564775863de0 0x5647758661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 4 client.27020 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T15:57:06.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 --> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564775731570 con 0x564775863de0 2024-06-02T15:57:06.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.953+0000 7fb436a00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564775863de0 0x5647758661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb436a00700 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564775863de0 0x5647758661d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8224ea20851865 server_cookie=14a084fc1a75874 in_seq=0 out_seq=0 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb421600700 10 client.27020 ms_handle_connect on v2:172.21.15.120:6838/3642613472 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb421600700 1 -- 172.21.15.90:0/1876247225 <== mds.0 v2:172.21.15.120:6838/3642613472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564775731570 con 0x564775863de0 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb421600700 10 client.27020 handle_command_reply: tid=0 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 client.27020 shutdown 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 msgr2=0x5647757e94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb430009e10 tx=0x7fb430009920 comp rx=0 tx=0).stop 2024-06-02T15:57:06.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 shutdown_connections 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:6838/3642613472,v1:172.21.15.120:6839/3642613472] conn(0x564775863de0 0x5647758661d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5647757dd7d0 0x5647757f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb421600700 0 client.27020 ms_handle_reset on v2:172.21.15.120:6838/3642613472 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5647757d7880 0x5647757e94d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 --2- 172.21.15.90:0/1876247225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647757dceb0 0x5647757e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 >> 172.21.15.90:0/1876247225 conn(0x564775705be0 msgr2=0x5647757df0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 shutdown_connections 2024-06-02T15:57:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.954+0000 7fb448fadb80 1 -- 172.21.15.90:0/1876247225 wait complete. 2024-06-02T15:57:06.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb448fadb80 20 client.27020 trim_cache size 0 max 16384 2024-06-02T15:57:06.960 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 msgr2=0x7fb41c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 0x7fb41c0b4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb4280aa310 tx=0x7fb4280e6df0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 msgr2=0x7fb43819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 0x7fb43819be40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb42800fa00 comp rx=0 tx=0).stop 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 shutdown_connections 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fb41c0b2900 0x7fb41c0b4dc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb43810ffa0 0x7fb4381a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb438075770 0x7fb43819be40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 --2- 172.21.15.90:0/2455066864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb438076090 0x7fb43819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.955+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 >> 172.21.15.90:0/2455066864 conn(0x7fb43807a6a0 msgr2=0x7fb438100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.956+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 shutdown_connections 2024-06-02T15:57:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:06.956+0000 7fb43e600700 1 -- 172.21.15.90:0/2455066864 wait complete. 2024-06-02T15:57:08.537 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_7aec00165b4b073a25396fd0279151d2b2fa23fe/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-02T15:57:08.542 INFO:tasks.fwd_scrub:done joining 2024-06-02T15:57:08.543 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-02T15:57:08.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-02T15:57:08.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:08 smithi090 ceph-mon[31595]: pgmap v899: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:08.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 -- 172.21.15.90:0/2543183282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88102aa0 msgr2=0x7faa88102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:08.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 --2- 172.21.15.90:0/2543183282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88102aa0 0x7faa88102ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa70009900 tx=0x7faa7002a400 comp rx=0 tx=0).stop 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 -- 172.21.15.90:0/2543183282 shutdown_connections 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 --2- 172.21.15.90:0/2543183282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88104650 0x7faa8810aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 --2- 172.21.15.90:0/2543183282 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88103c90 0x7faa88104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 --2- 172.21.15.90:0/2543183282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88102aa0 0x7faa88102ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 -- 172.21.15.90:0/2543183282 >> 172.21.15.90:0/2543183282 conn(0x7faa880fe030 msgr2=0x7faa88100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 -- 172.21.15.90:0/2543183282 shutdown_connections 2024-06-02T15:57:08.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.788+0000 7faa8f000640 1 -- 172.21.15.90:0/2543183282 wait complete. 2024-06-02T15:57:08.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.789+0000 7faa8f000640 1 Processor -- start 2024-06-02T15:57:08.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 -- start start 2024-06-02T15:57:08.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:08.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 0x7faa8819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:08.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88104650 0x7faa881a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:08.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa8810e5f0 con 0x7faa88102aa0 2024-06-02T15:57:08.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faa8810e470 con 0x7faa88104650 2024-06-02T15:57:08.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa8f000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa8810e770 con 0x7faa88103c90 2024-06-02T15:57:08.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa87e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:08.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa87e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53544/0 (socket says 172.21.15.90:53544) 2024-06-02T15:57:08.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.790+0000 7faa87e00640 1 -- 172.21.15.90:0/1405706383 learned_addr learned my addr 172.21.15.90:0/1405706383 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:08.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 0x7faa8819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa8d000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88104650 0x7faa881a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 -- 172.21.15.90:0/1405706383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88104650 msgr2=0x7faa881a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88104650 0x7faa881a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 -- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 msgr2=0x7faa8819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 -- 172.21.15.90:0/1405706383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa881a2b20 con 0x7faa88103c90 2024-06-02T15:57:08.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87e00640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.791+0000 7faa87400640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 0x7faa8819b440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa7800e9f0 tx=0x7faa7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.792+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7800cd80 con 0x7faa88103c90 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.792+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa780035f0 con 0x7faa88103c90 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.792+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78010670 con 0x7faa88103c90 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.792+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa881a2e10 con 0x7faa88103c90 2024-06-02T15:57:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.792+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa881a3280 con 0x7faa88103c90 2024-06-02T15:57:08.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.793+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa881a3510 con 0x7faa88103c90 2024-06-02T15:57:08.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.795+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faa780037a0 con 0x7faa88103c90 2024-06-02T15:57:08.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.797+0000 7faa84c00640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 0x7faa6c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:08.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.797+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7faa780e2820 con 0x7faa88103c90 2024-06-02T15:57:08.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.797+0000 7faa87e00640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 0x7faa6c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:08.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.798+0000 7faa87e00640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 0x7faa6c0b4cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa70000c00 tx=0x7faa70005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:08.799+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa780aa130 con 0x7faa88103c90 2024-06-02T15:57:08.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:08 smithi120 ceph-mon[36451]: pgmap v899: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:08.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:08 smithi135 ceph-mon[36839]: pgmap v899: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:09.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.009+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa881a3780 con 0x7faa88103c90 2024-06-02T15:57:09.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.011+0000 7faa84c00640 1 -- 172.21.15.90:0/1405706383 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24751 (secure 0 0 0) 0x7faa780a9a30 con 0x7faa88103c90 2024-06-02T15:57:09.009 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:57:09.009 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":94,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:54:20.318013+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:57:09.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 msgr2=0x7faa6c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 0x7faa6c0b4cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa70000c00 tx=0x7faa70005a20 comp rx=0 tx=0).stop 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 msgr2=0x7faa8819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 0x7faa8819b440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa7800e9f0 tx=0x7faa7800eeb0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 shutdown_connections 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7faa6c0b27f0 0x7faa6c0b4cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faa88104650 0x7faa881a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa88103c90 0x7faa8819b440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 --2- 172.21.15.90:0/1405706383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa88102aa0 0x7faa8819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 >> 172.21.15.90:0/1405706383 conn(0x7faa880fe030 msgr2=0x7faa88074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 shutdown_connections 2024-06-02T15:57:09.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.014+0000 7faa8f000640 1 -- 172.21.15.90:0/1405706383 wait complete. 2024-06-02T15:57:09.033 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T15:32:41.694560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-02T15:38:10.691399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-02T15:38:15.361392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-02T15:38:27.767622+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-02T15:57:09.033 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-02T15:57:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 -- 172.21.15.90:0/1805401947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c077410 msgr2=0x7f8c5c077830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 --2- 172.21.15.90:0/1805401947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c077410 0x7f8c5c077830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009860 tx=0x7f8c5002a340 comp rx=0 tx=0).stop 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 -- 172.21.15.90:0/1805401947 shutdown_connections 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 --2- 172.21.15.90:0/1805401947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c079890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 --2- 172.21.15.90:0/1805401947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c078650 0x7f8c5c078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 --2- 172.21.15.90:0/1805401947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c077410 0x7f8c5c077830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 -- 172.21.15.90:0/1805401947 >> 172.21.15.90:0/1805401947 conn(0x7f8c5c100100 msgr2=0x7f8c5c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 -- 172.21.15.90:0/1805401947 shutdown_connections 2024-06-02T15:57:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.243+0000 7f8c62800640 1 -- 172.21.15.90:0/1805401947 wait complete. 2024-06-02T15:57:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 Processor -- start 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 -- start start 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 0x7f8c5c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 0x7f8c5c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c114ad0 con 0x7f8c5c078ff0 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c114950 con 0x7f8c5c078650 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.245+0000 7f8c62800640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c114c50 con 0x7f8c5c077410 2024-06-02T15:57:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61400640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 0x7f8c5c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53556/0 (socket says 172.21.15.90:53556) 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 -- 172.21.15.90:0/1213839468 learned_addr learned my addr 172.21.15.90:0/1213839468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c60a00640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 0x7f8c5c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 -- 172.21.15.90:0/1213839468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 msgr2=0x7f8c5c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 0x7f8c5c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 -- 172.21.15.90:0/1213839468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 msgr2=0x7f8c5c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 0x7f8c5c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 -- 172.21.15.90:0/1213839468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c1a6dc0 con 0x7f8c5c078ff0 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c60a00640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 0x7f8c5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61400640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 0x7f8c5c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.246+0000 7f8c61e00640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009fd0 tx=0x7f8c5400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.247+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004b20 con 0x7f8c5c078ff0 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.247+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c54003e20 con 0x7f8c5c078ff0 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.247+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c1a70b0 con 0x7f8c5c078ff0 2024-06-02T15:57:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.247+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54004820 con 0x7f8c5c078ff0 2024-06-02T15:57:09.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.247+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c1a7600 con 0x7f8c5c078ff0 2024-06-02T15:57:09.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.248+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c111500 con 0x7f8c5c078ff0 2024-06-02T15:57:09.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.251+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8c5401d020 con 0x7f8c5c078ff0 2024-06-02T15:57:09.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.252+0000 7f8c5a000640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 0x7f8c440b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.252+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8c540e2640 con 0x7f8c5c078ff0 2024-06-02T15:57:09.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.252+0000 7f8c61400640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 0x7f8c440b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.253+0000 7f8c61400640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 0x7f8c440b4d30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8c50000c00 tx=0x7f8c50030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:09.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.257+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c54052df0 con 0x7f8c5c078ff0 2024-06-02T15:57:09.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.467+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c5c063650 con 0x7f8c5c078ff0 2024-06-02T15:57:09.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.468+0000 7f8c5a000640 1 -- 172.21.15.90:0/1213839468 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f8c54004de0 con 0x7f8c5c078ff0 2024-06-02T15:57:09.466 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-06-02T15:57:09.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 msgr2=0x7f8c440b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 0x7f8c440b4d30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8c50000c00 tx=0x7f8c50030470 comp rx=0 tx=0).stop 2024-06-02T15:57:09.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 msgr2=0x7f8c5c1a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009fd0 tx=0x7f8c5400c800 comp rx=0 tx=0).stop 2024-06-02T15:57:09.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 shutdown_connections 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f8c440b2870 0x7f8c440b4d30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c5c078ff0 0x7f8c5c1a6750 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c5c078650 0x7f8c5c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 --2- 172.21.15.90:0/1213839468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c077410 0x7f8c5c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 >> 172.21.15.90:0/1213839468 conn(0x7f8c5c100100 msgr2=0x7f8c5c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 shutdown_connections 2024-06-02T15:57:09.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.470+0000 7f8c62800640 1 -- 172.21.15.90:0/1213839468 wait complete. 2024-06-02T15:57:09.487 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-02T15:57:09.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:09 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1405706383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:57:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 -- 172.21.15.90:0/1816837256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b4076a10 msgr2=0x7f33b4074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1816837256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b4076a10 0x7f33b4074e60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009900 tx=0x7f33a402a490 comp rx=0 tx=0).stop 2024-06-02T15:57:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 -- 172.21.15.90:0/1816837256 shutdown_connections 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1816837256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b410a810 0x7f33b410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1816837256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4075430 0x7f33b4075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1816837256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b4076a10 0x7f33b4074e60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 -- 172.21.15.90:0/1816837256 >> 172.21.15.90:0/1816837256 conn(0x7f33b40fdef0 msgr2=0x7f33b4100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 -- 172.21.15.90:0/1816837256 shutdown_connections 2024-06-02T15:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.695+0000 7f33b9c00640 1 -- 172.21.15.90:0/1816837256 wait complete. 2024-06-02T15:57:09.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.696+0000 7f33b9c00640 1 Processor -- start 2024-06-02T15:57:09.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 -- start start 2024-06-02T15:57:09.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4076a10 0x7f33b419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b410a810 0x7f33b41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41106e0 con 0x7f33b410a810 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4110560 con 0x7f33b4075430 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9c00640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4110860 con 0x7f33b4076a10 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b3e00640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:47422/0 (socket says 172.21.15.90:47422) 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b410a810 0x7f33b41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b3e00640 1 -- 172.21.15.90:0/1850573067 learned_addr learned my addr 172.21.15.90:0/1850573067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.697+0000 7f33b3400640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4076a10 0x7f33b419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 -- 172.21.15.90:0/1850573067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4076a10 msgr2=0x7f33b419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4076a10 0x7f33b419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 -- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b410a810 msgr2=0x7f33b41a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b410a810 0x7f33b41a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 -- 172.21.15.90:0/1850573067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b41a29d0 con 0x7f33b4075430 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b3e00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005ab0 tx=0x7f33a4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a403f070 con 0x7f33b4075430 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a4031c50 con 0x7f33b4075430 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4035030 con 0x7f33b4075430 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.698+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b41a2c60 con 0x7f33b4075430 2024-06-02T15:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.699+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b41a3100 con 0x7f33b4075430 2024-06-02T15:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.700+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b41095d0 con 0x7f33b4075430 2024-06-02T15:57:09.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.703+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f33a4042070 con 0x7f33b4075430 2024-06-02T15:57:09.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.705+0000 7f33b0c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 0x7f33940b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:09.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.705+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f33a4103470 con 0x7f33b4075430 2024-06-02T15:57:09.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.705+0000 7f33b3400640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 0x7f33940b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:09.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.706+0000 7f33b3400640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 0x7f33940b4cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f339c009bd0 tx=0x7f339c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:09.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.706+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33a40ca790 con 0x7f33b4075430 2024-06-02T15:57:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:09 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1405706383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:57:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:09 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1405706383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:57:09.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.916+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f33b402de30 con 0x7f33b4075430 2024-06-02T15:57:09.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.917+0000 7f33b0c00640 1 -- 172.21.15.90:0/1850573067 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f33a40c4e20 con 0x7f33b4075430 2024-06-02T15:57:09.915 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 msgr2=0x7f33940b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 0x7f33940b4cb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f339c009bd0 tx=0x7f339c005f90 comp rx=0 tx=0).stop 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 msgr2=0x7f33b419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005ab0 tx=0x7f33a4003b90 comp rx=0 tx=0).stop 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 shutdown_connections 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f33940b27f0 0x7f33940b4cb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33b410a810 0x7f33b41a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33b4076a10 0x7f33b419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 --2- 172.21.15.90:0/1850573067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33b4075430 0x7f33b419adc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:09.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 >> 172.21.15.90:0/1850573067 conn(0x7f33b40fdef0 msgr2=0x7f33b41067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:09.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 shutdown_connections 2024-06-02T15:57:09.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:09.921+0000 7f33b9c00640 1 -- 172.21.15.90:0/1850573067 wait complete. 2024-06-02T15:57:09.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 -- 172.21.15.90:0/2400393275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 msgr2=0x7f533410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 --2- 172.21.15.90:0/2400393275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533410a600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f531c009ca0 tx=0x7f531c02a8b0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 -- 172.21.15.90:0/2400393275 shutdown_connections 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 --2- 172.21.15.90:0/2400393275 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 0x7f53341113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 --2- 172.21.15.90:0/2400393275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533410a600 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 --2- 172.21.15.90:0/2400393275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f53341093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 -- 172.21.15.90:0/2400393275 >> 172.21.15.90:0/2400393275 conn(0x7f5334077f20 msgr2=0x7f533407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:10.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 -- 172.21.15.90:0/2400393275 shutdown_connections 2024-06-02T15:57:10.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.146+0000 7f5339600640 1 -- 172.21.15.90:0/2400393275 wait complete. 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 Processor -- start 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 -- start start 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f533419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 0x7f53341a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5334114ad0 con 0x7f533410a180 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5334114950 con 0x7f533410ab40 2024-06-02T15:57:10.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.147+0000 7f5339600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5334114c50 con 0x7f5334108f90 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53588/0 (socket says 172.21.15.90:53588) 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 -- 172.21.15.90:0/3936317868 learned_addr learned my addr 172.21.15.90:0/3936317868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333e00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f533419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5338c00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 0x7f53341a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 -- 172.21.15.90:0/3936317868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 msgr2=0x7f533419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f533419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.148+0000 7f5333400640 1 -- 172.21.15.90:0/3936317868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 msgr2=0x7f53341a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5333400640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 0x7f53341a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5333400640 1 -- 172.21.15.90:0/3936317868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53341a6d60 con 0x7f533410a180 2024-06-02T15:57:10.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5333e00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f533419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T15:57:10.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5333400640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f531c02add0 tx=0x7f531c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c004240 con 0x7f533410a180 2024-06-02T15:57:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.149+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f531c040070 con 0x7f533410a180 2024-06-02T15:57:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.150+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c031d70 con 0x7f533410a180 2024-06-02T15:57:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.150+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53341a6ff0 con 0x7f533410a180 2024-06-02T15:57:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.150+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53341a7460 con 0x7f533410a180 2024-06-02T15:57:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.151+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5334110330 con 0x7f533410a180 2024-06-02T15:57:10.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.153+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f531c0314d0 con 0x7f533410a180 2024-06-02T15:57:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.155+0000 7f5330c00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 0x7f53180b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.155+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f531c103ac0 con 0x7f533410a180 2024-06-02T15:57:10.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.155+0000 7f5333e00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 0x7f53180b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.156+0000 7f5333e00640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 0x7f53180b4d30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f53240058c0 tx=0x7f5324005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:10.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.158+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f531c0cb3d0 con 0x7f533410a180 2024-06-02T15:57:10.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.367+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f533402de10 con 0x7f533410a180 2024-06-02T15:57:10.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.368+0000 7f5330c00640 1 -- 172.21.15.90:0/3936317868 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f531c0cacd0 con 0x7f533410a180 2024-06-02T15:57:10.365 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-06-02T15:57:10.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 msgr2=0x7f53180b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 0x7f53180b4d30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f53240058c0 tx=0x7f5324005850 comp rx=0 tx=0).stop 2024-06-02T15:57:10.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 msgr2=0x7f533419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f531c02add0 tx=0x7f531c003ba0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 shutdown_connections 2024-06-02T15:57:10.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f53180b2870 0x7f53180b4d30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f533410ab40 0x7f53341a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f533410a180 0x7f533419f6d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 --2- 172.21.15.90:0/3936317868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5334108f90 0x7f533419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 >> 172.21.15.90:0/3936317868 conn(0x7f5334077f20 msgr2=0x7f533410d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:10.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 shutdown_connections 2024-06-02T15:57:10.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.371+0000 7f5339600640 1 -- 172.21.15.90:0/3936317868 wait complete. 2024-06-02T15:57:10.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-02T15:57:10.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 -- 172.21.15.90:0/947294920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc41035b0 msgr2=0x7f0cc41059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 --2- 172.21.15.90:0/947294920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc41035b0 0x7f0cc41059a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009900 tx=0x7f0cb402a400 comp rx=0 tx=0).stop 2024-06-02T15:57:10.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 -- 172.21.15.90:0/947294920 shutdown_connections 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 --2- 172.21.15.90:0/947294920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 --2- 172.21.15.90:0/947294920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc4105ee0 0x7f0cc41082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 --2- 172.21.15.90:0/947294920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc41035b0 0x7f0cc41059a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 -- 172.21.15.90:0/947294920 >> 172.21.15.90:0/947294920 conn(0x7f0cc40fcf00 msgr2=0x7f0cc40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 -- 172.21.15.90:0/947294920 shutdown_connections 2024-06-02T15:57:10.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.590+0000 7f0ccc000640 1 -- 172.21.15.90:0/947294920 wait complete. 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.591+0000 7f0ccc000640 1 Processor -- start 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.591+0000 7f0ccc000640 1 -- start start 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc41035b0 0x7f0cc4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc4105ee0 0x7f0cc4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410e5e0 con 0x7f0cc4108810 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410e460 con 0x7f0cc4105ee0 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0ccc000640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410e760 con 0x7f0cc41035b0 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53610/0 (socket says 172.21.15.90:53610) 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 -- 172.21.15.90:0/2609857229 learned_addr learned my addr 172.21.15.90:0/2609857229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cc9600640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc41035b0 0x7f0cc4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cc8c00640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc4105ee0 0x7f0cc4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 -- 172.21.15.90:0/2609857229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc41035b0 msgr2=0x7f0cc4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc41035b0 0x7f0cc4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 -- 172.21.15.90:0/2609857229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc4105ee0 msgr2=0x7f0cc4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc4105ee0 0x7f0cc4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.592+0000 7f0cca000640 1 -- 172.21.15.90:0/2609857229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc41a0960 con 0x7f0cc4108810 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0cca000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00e970 tx=0x7f0cbc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00ccc0 con 0x7f0cc4108810 2024-06-02T15:57:10.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cbc00ce60 con 0x7f0cc4108810 2024-06-02T15:57:10.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc41a0c50 con 0x7f0cc4108810 2024-06-02T15:57:10.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc010430 con 0x7f0cc4108810 2024-06-02T15:57:10.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.593+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc41a1150 con 0x7f0cc4108810 2024-06-02T15:57:10.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.594+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc40fe9f0 con 0x7f0cc4108810 2024-06-02T15:57:10.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.597+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0cbc004ff0 con 0x7f0cc4108810 2024-06-02T15:57:10.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.598+0000 7f0cc2000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 0x7f0ca80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:10.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.598+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f0cbc0e2a00 con 0x7f0cc4108810 2024-06-02T15:57:10.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.599+0000 7f0cc9600640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 0x7f0ca80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:10.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.599+0000 7f0cc9600640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 0x7f0ca80b4cb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:10.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.603+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cbc0aa310 con 0x7f0cc4108810 2024-06-02T15:57:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:10 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1213839468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:10 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1850573067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:10 smithi120 ceph-mon[36451]: pgmap v900: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:10 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3936317868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.810+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0cc40636f0 con 0x7f0cc4108810 2024-06-02T15:57:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.811+0000 7f0cc2000640 1 -- 172.21.15.90:0/2609857229 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f0cbc0a9c10 con 0x7f0cc4108810 2024-06-02T15:57:10.808 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-06-02T15:57:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.813+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 msgr2=0x7f0ca80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.813+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 0x7f0ca80b4cb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb40314a0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 msgr2=0x7f0cc41a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00e970 tx=0x7f0cbc00ed70 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 shutdown_connections 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f0ca80b27f0 0x7f0ca80b4cb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc4108810 0x7f0cc41a02f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0cc4105ee0 0x7f0cc4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 --2- 172.21.15.90:0/2609857229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cc41035b0 0x7f0cc4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 >> 172.21.15.90:0/2609857229 conn(0x7f0cc40fcf00 msgr2=0x7f0cc4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 shutdown_connections 2024-06-02T15:57:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:10.814+0000 7f0ccc000640 1 -- 172.21.15.90:0/2609857229 wait complete. 2024-06-02T15:57:10.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:10 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1213839468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:10 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1850573067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:10 smithi135 ceph-mon[36839]: pgmap v900: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:10.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:10 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3936317868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:10 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1213839468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:10 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1850573067' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:10 smithi090 ceph-mon[31595]: pgmap v900: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:10 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3936317868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T15:57:10.829 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-02T15:57:11.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.031+0000 7f8112600640 1 -- 172.21.15.90:0/4045380082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 msgr2=0x7f810c106140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:11.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.031+0000 7f8112600640 1 --2- 172.21.15.90:0/4045380082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c106140 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009900 tx=0x7f80f402a3f0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 -- 172.21.15.90:0/4045380082 shutdown_connections 2024-06-02T15:57:11.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 --2- 172.21.15.90:0/4045380082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 --2- 172.21.15.90:0/4045380082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c106140 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 --2- 172.21.15.90:0/4045380082 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 -- 172.21.15.90:0/4045380082 >> 172.21.15.90:0/4045380082 conn(0x7f810c100060 msgr2=0x7f810c102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:11.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 -- 172.21.15.90:0/4045380082 shutdown_connections 2024-06-02T15:57:11.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.032+0000 7f8112600640 1 -- 172.21.15.90:0/4045380082 wait complete. 2024-06-02T15:57:11.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 Processor -- start 2024-06-02T15:57:11.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 -- start start 2024-06-02T15:57:11.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:11.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1105e0 con 0x7f810c106680 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f810c110460 con 0x7f810c104ad0 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.033+0000 7f8112600640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f810c110760 con 0x7f810c105cc0 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:11.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44824/0 (socket says 172.21.15.90:44824) 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111c00640 1 -- 172.21.15.90:0/2563485751 learned_addr learned my addr 172.21.15.90:0/2563485751 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8103e00640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 -- 172.21.15.90:0/2563485751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 msgr2=0x7f810c19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 -- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 msgr2=0x7f810c1a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 -- 172.21.15.90:0/2563485751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1a2910 con 0x7f810c104ad0 2024-06-02T15:57:11.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111c00640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.034+0000 7f8111200640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c19ad00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00e9c0 tx=0x7f80fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.035+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00ccc0 con 0x7f810c104ad0 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.035+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80fc0035f0 con 0x7f810c104ad0 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.035+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c1a2c00 con 0x7f810c104ad0 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.035+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00ce60 con 0x7f810c104ad0 2024-06-02T15:57:11.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.035+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1a3150 con 0x7f810c104ad0 2024-06-02T15:57:11.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.036+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c10bf10 con 0x7f810c104ad0 2024-06-02T15:57:11.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.038+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f80fc004000 con 0x7f810c104ad0 2024-06-02T15:57:11.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.040+0000 7f810a000640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 0x7f80f00b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:11.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.040+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f80fc0e2b70 con 0x7f810c104ad0 2024-06-02T15:57:11.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.040+0000 7f8103e00640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 0x7f80f00b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:11.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.041+0000 7f8103e00640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 0x7f80f00b4cb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000c00 tx=0x7f80f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:11.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.042+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80fc0aa480 con 0x7f810c104ad0 2024-06-02T15:57:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.282+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f810c02dba0 con 0x7f810c104ad0 2024-06-02T15:57:11.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.295+0000 7f810a000640 1 -- 172.21.15.90:0/2563485751 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815639 (secure 0 0 0) 0x7f80fc0a9d80 con 0x7f810c104ad0 2024-06-02T15:57:11.305 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 18, 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "active_gid": 14152, 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "active_name": "x", 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addrs": { 2024-06-02T15:57:11.306 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6800", 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 436264481 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6801", 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 436264481 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addr": "172.21.15.90:6801/436264481", 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: "active_change": "2024-06-02T15:29:57.532295+0000", 2024-06-02T15:57:11.308 INFO:teuthology.orchestra.run.smithi090.stdout: "active_mgr_features": 4540701547738038271, 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "available": true, 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "standbys": [ 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24109, 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-06-02T15:57:11.309 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:11.310 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.313 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.317 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-06-02T15:57:11.324 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-06-02T15:57:11.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-06-02T15:57:11.326 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-06-02T15:57:11.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.333 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.335 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.337 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-06-02T15:57:11.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:11.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T15:57:11.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-06-02T15:57:11.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.351 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.352 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.353 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.357 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-06-02T15:57:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T15:57:11.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-06-02T15:57:11.363 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.364 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:11.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T15:57:11.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.368 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:11.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.371 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.373 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-06-02T15:57:11.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.379 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-06-02T15:57:11.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T15:57:11.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T15:57:11.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-06-02T15:57:11.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T15:57:11.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-06-02T15:57:11.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-06-02T15:57:11.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-06-02T15:57:11.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-06-02T15:57:11.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.409 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-06-02T15:57:11.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-06-02T15:57:11.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-06-02T15:57:11.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-06-02T15:57:11.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-06-02T15:57:11.418 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-06-02T15:57:11.419 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.421 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-06-02T15:57:11.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-06-02T15:57:11.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.428 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.431 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-06-02T15:57:11.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-06-02T15:57:11.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.437 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.445 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.447 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.452 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-06-02T15:57:11.453 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-06-02T15:57:11.461 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.462 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.463 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-06-02T15:57:11.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.465 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-06-02T15:57:11.466 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.470 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.471 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-06-02T15:57:11.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-06-02T15:57:11.478 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-06-02T15:57:11.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T15:57:11.481 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.485 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.494 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T15:57:11.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-06-02T15:57:11.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.513 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T15:57:11.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.516 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.519 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.520 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.525 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T15:57:11.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.534 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T15:57:11.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-06-02T15:57:11.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T15:57:11.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.550 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T15:57:11.555 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.556 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.558 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.560 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T15:57:11.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.564 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-06-02T15:57:11.566 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.567 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.568 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:11.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.572 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T15:57:11.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-06-02T15:57:11.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.581 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-06-02T15:57:11.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.586 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.590 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.593 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.596 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.599 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.604 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:11.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.611 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:11.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.614 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:11.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.619 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.620 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.624 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T15:57:11.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-06-02T15:57:11.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.635 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-06-02T15:57:11.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.638 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.639 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-06-02T15:57:11.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.646 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.647 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.649 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.653 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-06-02T15:57:11.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.663 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-06-02T15:57:11.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T15:57:11.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.666 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.669 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.672 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.676 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.681 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.685 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:11.686 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.688 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.689 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.694 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.699 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.700 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-06-02T15:57:11.704 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.705 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.706 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.710 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.714 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.715 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.717 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-06-02T15:57:11.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.719 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.722 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.723 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.725 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.726 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.730 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-06-02T15:57:11.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.732 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.733 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.735 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.738 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.739 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-06-02T15:57:11.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.742 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-06-02T15:57:11.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-06-02T15:57:11.745 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.746 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-06-02T15:57:11.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T15:57:11.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.753 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.755 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.759 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.764 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.766 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.767 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.769 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.770 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.771 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.772 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.773 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.775 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.778 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.782 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.785 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.786 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.787 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.788 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.791 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.792 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.795 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.802 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.803 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.805 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.806 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.807 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.809 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.812 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.813 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.816 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.817 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.820 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.821 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.823 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.824 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.825 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.826 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.827 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.828 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.830 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.831 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.832 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.834 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.835 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:11.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.840 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-06-02T15:57:11.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.845 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.846 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.847 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.848 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.849 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.850 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.851 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.852 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.853 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-06-02T15:57:11.854 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.855 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.857 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.859 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.862 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-06-02T15:57:11.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.864 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.865 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.866 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.867 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.868 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.870 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.872 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.876 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-06-02T15:57:11.878 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-06-02T15:57:11.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.885 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.886 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:11.888 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.889 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.890 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:11.891 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.894 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.898 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.899 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.900 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.901 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-06-02T15:57:11.902 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.903 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.904 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.906 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.907 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.908 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.910 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.911 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.912 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.913 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.914 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.919 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.920 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.921 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:11.922 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.927 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.928 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.930 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.932 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.933 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.934 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.936 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.937 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.938 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.939 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.940 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.941 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.942 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.943 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.944 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.945 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.946 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.948 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.949 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.951 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.952 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.957 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.961 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-06-02T15:57:11.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.967 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-06-02T15:57:11.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-06-02T15:57:11.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.978 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.979 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.981 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.982 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.983 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:11.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.987 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.988 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:11.989 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:11.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-06-02T15:57:11.994 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.995 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:11.996 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.997 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:11.998 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:11.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.001 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.004 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.008 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.009 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.010 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.011 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.014 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.017 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.024 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.028 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-06-02T15:57:12.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.035 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.036 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.039 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.043 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.044 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-06-02T15:57:12.048 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.049 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.050 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.051 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.052 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-06-02T15:57:12.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T15:57:12.055 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.056 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-06-02T15:57:12.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.062 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.063 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.063 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:12.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:12.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.068 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-06-02T15:57:12.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.070 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-06-02T15:57:12.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.075 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.076 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.077 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.078 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-06-02T15:57:12.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-06-02T15:57:12.090 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.092 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.093 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.096 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.099 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.100 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.101 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.108 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-06-02T15:57:12.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-06-02T15:57:12.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-06-02T15:57:12.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.120 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-06-02T15:57:12.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-06-02T15:57:12.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.128 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.132 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-06-02T15:57:12.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24113, 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "y", 2024-06-02T15:57:12.143 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.147 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.148 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.154 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-06-02T15:57:12.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.162 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-06-02T15:57:12.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.170 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-06-02T15:57:12.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:12.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-06-02T15:57:12.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T15:57:12.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.178 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-06-02T15:57:12.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-06-02T15:57:12.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T15:57:12.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.186 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.190 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.191 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.192 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:12.195 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.196 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.206 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-06-02T15:57:12.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T15:57:12.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-06-02T15:57:12.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T15:57:12.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.215 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T15:57:12.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-06-02T15:57:12.218 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.222 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-06-02T15:57:12.227 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T15:57:12.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-06-02T15:57:12.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.237 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.238 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-06-02T15:57:12.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.240 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-06-02T15:57:12.241 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.242 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T15:57:12.245 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.247 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-06-02T15:57:12.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-06-02T15:57:12.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-06-02T15:57:12.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T15:57:12.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-06-02T15:57:12.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.260 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T15:57:12.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.265 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-06-02T15:57:12.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.273 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.275 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T15:57:12.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.279 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.280 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.281 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.282 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.283 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T15:57:12.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T15:57:12.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.288 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-06-02T15:57:12.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-06-02T15:57:12.291 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-06-02T15:57:12.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-06-02T15:57:12.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T15:57:12.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.300 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-06-02T15:57:12.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T15:57:12.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.316 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.318 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T15:57:12.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.322 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.324 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.328 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.331 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.332 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.333 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.335 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.336 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.337 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.339 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T15:57:12.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T15:57:12.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T15:57:12.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.351 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.352 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.353 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.354 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.356 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.357 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T15:57:12.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.362 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:12.363 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.364 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-06-02T15:57:12.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-06-02T15:57:12.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.378 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.379 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.381 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.382 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.383 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.384 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T15:57:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T15:57:12.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T15:57:12.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.396 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.397 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T15:57:12.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T15:57:12.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.402 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T15:57:12.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.406 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.408 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.409 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-06-02T15:57:12.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T15:57:12.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.420 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:12.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.426 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.428 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.432 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.433 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.436 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T15:57:12.437 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-06-02T15:57:12.441 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.442 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:12.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-06-02T15:57:12.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.449 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:12.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.451 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.452 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.453 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-06-02T15:57:12.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-06-02T15:57:12.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.457 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.458 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.461 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.462 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.465 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-06-02T15:57:12.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.475 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.476 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.476 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.476 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.477 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.478 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.480 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.482 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.486 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.487 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-06-02T15:57:12.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.496 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-06-02T15:57:12.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-06-02T15:57:12.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.505 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.506 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.510 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.515 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-06-02T15:57:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-06-02T15:57:12.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T15:57:12.521 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-06-02T15:57:12.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.527 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.530 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.532 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.538 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.543 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.547 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.548 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-06-02T15:57:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.558 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.562 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.563 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T15:57:12.565 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.566 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.567 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.568 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.571 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-06-02T15:57:12.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-06-02T15:57:12.575 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-06-02T15:57:12.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-06-02T15:57:12.580 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.584 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.585 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.587 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.588 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.589 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.591 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.592 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.598 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.602 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-06-02T15:57:12.607 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.609 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.610 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.613 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.617 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.620 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.626 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.633 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.635 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.639 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.645 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.647 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.649 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.650 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.656 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.659 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.660 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.661 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.662 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:12.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.675 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.679 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-06-02T15:57:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.685 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-06-02T15:57:12.689 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.692 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.693 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.699 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.701 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-06-02T15:57:12.706 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.707 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.708 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.709 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.711 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.715 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.719 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.721 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.725 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.728 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T15:57:12.730 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.733 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.734 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.735 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-06-02T15:57:12.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.737 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.741 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.745 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.748 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.750 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.751 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.754 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.755 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.756 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.757 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.758 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.759 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.760 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.761 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.763 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.765 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.766 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.767 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.769 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.770 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.771 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.772 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.774 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.775 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-06-02T15:57:12.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.777 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.779 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.780 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.781 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.783 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.785 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.786 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.788 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-06-02T15:57:12.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.791 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.795 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.797 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.800 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.801 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-06-02T15:57:12.802 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.807 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.809 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.810 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-06-02T15:57:12.811 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.812 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.813 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.815 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.816 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.817 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.818 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.819 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.820 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.822 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.823 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.826 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.827 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.833 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.834 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.835 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.838 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.839 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.841 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.845 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.846 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.847 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.848 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.850 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.851 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.854 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.855 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.856 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.859 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.860 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.861 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.863 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.866 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.869 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.870 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.871 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.872 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.873 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.874 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.875 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.876 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-06-02T15:57:12.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T15:57:12.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.880 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.883 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.884 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-06-02T15:57:12.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.887 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.888 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.889 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.891 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.892 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-06-02T15:57:12.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-06-02T15:57:12.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.897 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.900 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.901 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.902 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.905 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.907 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.910 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-06-02T15:57:12.914 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.916 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.917 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.918 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.919 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.920 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.921 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.922 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.924 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.926 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.928 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.930 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.931 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.932 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.933 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.934 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.940 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.941 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T15:57:12.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.943 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.944 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.945 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.946 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.948 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.949 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.950 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.951 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.952 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.953 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.954 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.957 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.958 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-06-02T15:57:12.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.963 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.964 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.968 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "modules": [ 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard", 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "iostat", 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "nfs", 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "restful" 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.969 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:12.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.971 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.972 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.974 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:12.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:12.978 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-06-02T15:57:12.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-06-02T15:57:12.982 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.983 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.986 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-06-02T15:57:12.988 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-06-02T15:57:12.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.990 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.993 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.994 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:12.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.996 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:12.997 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-06-02T15:57:12.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:12.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-06-02T15:57:13.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T15:57:13.001 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:13.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-06-02T15:57:13.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.006 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-06-02T15:57:13.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.008 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-06-02T15:57:13.009 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.010 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.011 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.014 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.017 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.018 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-06-02T15:57:13.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T15:57:13.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-06-02T15:57:13.024 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.025 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:13.027 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T15:57:13.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.030 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.031 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.033 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-06-02T15:57:13.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T15:57:13.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-06-02T15:57:13.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T15:57:13.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.043 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.044 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-06-02T15:57:13.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.046 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.047 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.048 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.049 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.050 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.051 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.052 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.053 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.055 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T15:57:13.056 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-06-02T15:57:13.057 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-06-02T15:57:13.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.063 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-06-02T15:57:13.064 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-06-02T15:57:13.065 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-06-02T15:57:13.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.074 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T15:57:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-06-02T15:57:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T15:57:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-06-02T15:57:13.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T15:57:13.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.086 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.088 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:13.090 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.091 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-06-02T15:57:13.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-06-02T15:57:13.094 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.095 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.100 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.106 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.110 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-06-02T15:57:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-06-02T15:57:13.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-06-02T15:57:13.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T15:57:13.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-06-02T15:57:13.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T15:57:13.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-06-02T15:57:13.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.140 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-06-02T15:57:13.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.144 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.145 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.146 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.147 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.148 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.160 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T15:57:13.163 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.168 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T15:57:13.170 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.171 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.175 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T15:57:13.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T15:57:13.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T15:57:13.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T15:57:13.192 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T15:57:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-06-02T15:57:13.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T15:57:13.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.207 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.211 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T15:57:13.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T15:57:13.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.220 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T15:57:13.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T15:57:13.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.233 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.234 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-06-02T15:57:13.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-06-02T15:57:13.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.248 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:13.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.253 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.254 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.257 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-06-02T15:57:13.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:13.264 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-06-02T15:57:13.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:13.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.286 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-06-02T15:57:13.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.291 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-06-02T15:57:13.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-06-02T15:57:13.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.298 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.302 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.304 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.306 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.310 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.313 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.314 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-06-02T15:57:13.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.320 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-06-02T15:57:13.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T15:57:13.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:13.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T15:57:13.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.330 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.333 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.334 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.335 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-06-02T15:57:13.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.337 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.340 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.341 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-06-02T15:57:13.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.343 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.348 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.351 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.356 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.357 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.361 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.363 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.365 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.367 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.372 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-06-02T15:57:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.378 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.382 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.383 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.391 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.392 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.397 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-06-02T15:57:13.401 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.402 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.405 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.406 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-06-02T15:57:13.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.408 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-06-02T15:57:13.409 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.411 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.412 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.416 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.420 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.426 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.426 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.429 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.433 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.433 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.433 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.433 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.435 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-06-02T15:57:13.436 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.436 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.437 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.437 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.437 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.437 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.439 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.440 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.441 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.442 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.442 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.444 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.445 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.445 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.446 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.446 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.446 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.446 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.446 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.447 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.448 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.450 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.450 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.451 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.454 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.454 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.454 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.454 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.455 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:13.455 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.455 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.455 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.456 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-06-02T15:57:13.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-06-02T15:57:13.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T15:57:13.457 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.460 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.460 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.460 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.460 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.460 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.461 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.461 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.462 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.462 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.465 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.467 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.467 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.467 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.468 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.468 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.468 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.468 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.469 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.475 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.475 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.476 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.476 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.476 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.477 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.477 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.478 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.478 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.479 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.480 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.480 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.481 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.481 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.482 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.486 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.487 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.489 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.489 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.489 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.489 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.489 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.490 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.490 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.491 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.493 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.494 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.496 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.497 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.497 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-06-02T15:57:13.497 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.498 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.498 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.498 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.500 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.500 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.501 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.501 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.501 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.501 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.503 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.506 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.506 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.506 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.507 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.508 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.512 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.518 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.518 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.518 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.518 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-06-02T15:57:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.521 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.522 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.522 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.523 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.527 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.527 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.528 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.528 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.528 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.528 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.528 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.529 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.532 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:13.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.535 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.538 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-06-02T15:57:13.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-06-02T15:57:13.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-06-02T15:57:13.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-06-02T15:57:13.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.540 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.541 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.542 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-06-02T15:57:13.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-06-02T15:57:13.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.543 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.546 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.548 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.549 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.549 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.549 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.553 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.554 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.554 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.554 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.555 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.555 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.556 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.556 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.556 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.556 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.556 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.557 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.559 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.560 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-06-02T15:57:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-06-02T15:57:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-06-02T15:57:13.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.562 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-06-02T15:57:13.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.564 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-06-02T15:57:13.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-06-02T15:57:13.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-06-02T15:57:13.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-06-02T15:57:13.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.567 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.567 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.569 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.570 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.570 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.571 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-06-02T15:57:13.571 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.571 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.571 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.572 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-06-02T15:57:13.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-06-02T15:57:13.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.574 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T15:57:13.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T15:57:13.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.580 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.584 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.584 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.586 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.586 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.586 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.587 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.587 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.587 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.587 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.588 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.590 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.590 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.590 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.591 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.591 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-06-02T15:57:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-06-02T15:57:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.594 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T15:57:13.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.597 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-06-02T15:57:13.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-06-02T15:57:13.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-06-02T15:57:13.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.599 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:13.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:13.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.601 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-06-02T15:57:13.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T15:57:13.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-06-02T15:57:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.604 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-06-02T15:57:13.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-06-02T15:57:13.612 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.614 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.618 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.619 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-06-02T15:57:13.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-06-02T15:57:13.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.634 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.636 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.637 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.638 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.642 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.645 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.646 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.647 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.649 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.653 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.654 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.657 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.658 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.660 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.664 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-06-02T15:57:13.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.673 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.674 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.677 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.679 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-06-02T15:57:13.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.685 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.686 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-06-02T15:57:13.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.689 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.692 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.694 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-06-02T15:57:13.695 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.699 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.700 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-06-02T15:57:13.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.703 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.705 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.707 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.709 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.710 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.719 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.721 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.726 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.728 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.730 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.731 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.731 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.739 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.740 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.741 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.744 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.748 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-06-02T15:57:13.751 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.753 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.754 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.755 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.756 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.759 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.760 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.761 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.765 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-06-02T15:57:13.766 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.767 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.767 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.767 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-06-02T15:57:13.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-06-02T15:57:13.767 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:13.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:13.770 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.770 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.770 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.771 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.772 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.773 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.774 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.776 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.779 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.784 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.785 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-06-02T15:57:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.787 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.788 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T15:57:13.788 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.788 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.788 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.790 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-06-02T15:57:13.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.793 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T15:57:13.793 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.794 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T15:57:13.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T15:57:13.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.798 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.799 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.800 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.800 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.805 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-06-02T15:57:13.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-06-02T15:57:13.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.807 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.807 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.807 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.807 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-06-02T15:57:13.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.809 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.812 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.813 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.814 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.815 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.816 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.817 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.818 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.819 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.820 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.821 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.822 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.823 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.824 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.825 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.826 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.826 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.826 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.826 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.827 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.828 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.830 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.833 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.834 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-06-02T15:57:13.835 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.837 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.840 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.841 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.842 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.843 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.848 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.849 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.850 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.851 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.852 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.854 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.854 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.854 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.856 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.859 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.861 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-06-02T15:57:13.862 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.863 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.864 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-06-02T15:57:13.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T15:57:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.868 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-06-02T15:57:13.871 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-06-02T15:57:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.874 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-06-02T15:57:13.875 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.876 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-06-02T15:57:13.877 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.878 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.879 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-06-02T15:57:13.880 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-06-02T15:57:13.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.883 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-06-02T15:57:13.884 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-06-02T15:57:13.886 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-06-02T15:57:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.890 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-06-02T15:57:13.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-06-02T15:57:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-06-02T15:57:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: "services": { 2024-06-02T15:57:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard": "https://172.21.15.90:8443/" 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "always_on_modules": { 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "octopus": [ 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-06-02T15:57:13.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "pacific": [ 2024-06-02T15:57:13.898 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-06-02T15:57:13.899 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "quincy": [ 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-06-02T15:57:13.900 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-06-02T15:57:13.901 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "reef": [ 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-06-02T15:57:13.902 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-06-02T15:57:13.903 INFO:teuthology.orchestra.run.smithi090.stdout: "squid": [ 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-06-02T15:57:13.904 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: "last_failure_osd_epoch": 3, 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: "active_clients": [ 2024-06-02T15:57:13.905 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1527619074 2024-06-02T15:57:13.906 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "libcephsqlite", 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-06-02T15:57:13.907 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 742359897 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4175344633 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.909 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-06-02T15:57:13.910 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3327243880 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-06-02T15:57:13.911 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:11 smithi135 ceph-mon[36839]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2609857229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:11 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2563485751' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:12 smithi135 ceph-mon[36839]: pgmap v901: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:11 smithi120 ceph-mon[36451]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:13.913 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2609857229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:57:13.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:11 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2563485751' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T15:57:13.914 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:12 smithi120 ceph-mon[36451]: pgmap v901: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:13.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 msgr2=0x7f80f00b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:13.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 0x7f80f00b4cb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f80f4000c00 tx=0x7f80f40314a0 comp rx=0 tx=0).stop 2024-06-02T15:57:13.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 msgr2=0x7f810c19ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:13.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c19ad00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00e9c0 tx=0x7f80fc00ee80 comp rx=0 tx=0).stop 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 shutdown_connections 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f80f00b27f0 0x7f80f00b4cb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810c106680 0x7f810c1a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f810c105cc0 0x7f810c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 --2- 172.21.15.90:0/2563485751 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c104ad0 0x7f810c19ad00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 >> 172.21.15.90:0/2563485751 conn(0x7f810c100060 msgr2=0x7f810c1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 shutdown_connections 2024-06-02T15:57:13.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:11.306+0000 7f8112600640 1 -- 172.21.15.90:0/2563485751 wait complete. 2024-06-02T15:57:13.925 DEBUG:tasks.check_counter:Getting stats from a 2024-06-02T15:57:13.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-02T15:57:13.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:11 smithi090 ceph-mon[31595]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 3 up:standby-replay 6 up:standby 2024-06-02T15:57:13.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2609857229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-02T15:57:13.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:11 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2563485751' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T15:57:13.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:12 smithi090 ceph-mon[31595]: pgmap v901: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:14.369 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:14.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:14 smithi090 ceph-mon[31595]: pgmap v902: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:14.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:14 smithi120 ceph-mon[36451]: pgmap v902: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:14.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:14 smithi135 ceph-mon[36839]: pgmap v902: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:16.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:16 smithi090 ceph-mon[31595]: pgmap v903: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:16.618 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:16 smithi135 ceph-mon[36839]: pgmap v903: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:16.725 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:57:16.725 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:16.725 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 88621, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 72057, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 43986896, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 133382611, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 24, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 13.062544334, 2024-06-02T15:57:16.726 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 5.934315240, 2024-06-02T15:57:16.727 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 6.095246592, 2024-06-02T15:57:16.727 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 2.657235147, 2024-06-02T15:57:16.727 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:16.727 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 72058, 2024-06-02T15:57:16.728 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 13.490665346, 2024-06-02T15:57:16.728 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000187219 2024-06-02T15:57:16.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.728 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:16.728 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 122752, 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.047351008, 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000385 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 666208, 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 307552 2024-06-02T15:57:16.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 24257, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 24492, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 8001642, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 78873275, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 15, 2024-06-02T15:57:16.730 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 5.806553226, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 1.476455838, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.685678165, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 3.257877495, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 24490, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 8.227099329, 2024-06-02T15:57:16.731 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000335937 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 6, 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000864, 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000144 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 474256 2024-06-02T15:57:16.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 119509, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 212266, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 42827148, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 187281535, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 14, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 7, 2024-06-02T15:57:16.733 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 37.145702269, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 25.026898828, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 29.717652885, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 3.787119453, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 212263, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 39.279208117, 2024-06-02T15:57:16.734 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000185049 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 229723, 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.096494732, 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000420 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.735 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 4032, 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 1344 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-06-02T15:57:16.736 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 14524, 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 122.000245969, 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.008399906 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-06-02T15:57:16.737 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 28219, 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 17.270735663, 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000612025 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 65460, 2024-06-02T15:57:16.738 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 57354, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 57354, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 300.146226016, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.005233222 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 8106, 2024-06-02T15:57:16.739 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 199, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 983, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 112, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 76, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 20, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 3, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 2, 2024-06-02T15:57:16.740 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 15, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 15, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 41502, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 1, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 1, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 4, 2024-06-02T15:57:16.741 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 89309, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 75527, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 8070, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 2, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 198, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:16.742 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 315, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 30566, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 425, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 22139, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:16.743 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 27552, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 1, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 27552, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:16.744 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 8, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 8, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 27544, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 6, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 329, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 38624, 2024-06-02T15:57:16.745 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 19844, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 19530, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 1913, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 29100 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.746 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 4, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 159, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:16.747 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 18511, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 18511, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-06-02T15:57:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 334, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 193, 2024-06-02T15:57:16.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 3, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 78486, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-06-02T15:57:16.750 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 78486, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 99, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 99, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 8, 2024-06-02T15:57:16.751 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 185478493, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2436, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 389.114852269, 2024-06-02T15:57:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.159735161 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 25, 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 30727, 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 30702, 2024-06-02T15:57:16.753 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 23, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 2532, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 2509, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 20, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 60112, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 60092, 2024-06-02T15:57:16.754 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 1, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 47817, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 47816, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 169576, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 223608 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 76725, 2024-06-02T15:57:16.755 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 11604, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 65460, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 123, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 26266, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 19799, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 112.516161491, 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.005682921 2024-06-02T15:57:16.756 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 291, 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 1.437807895, 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.004940920 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.757 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.758 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 17844, 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 58.067234576, 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.003254160 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.759 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000092561, 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000092561 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.760 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.761 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.762 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.762 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:16.762 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.762 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.763 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.763 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-06-02T15:57:16.763 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.763 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.764 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.765 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-06-02T15:57:16.766 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 908, 2024-06-02T15:57:16.767 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 13.538364997, 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.014910093 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.769 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:16.769 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.769 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.769 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1, 2024-06-02T15:57:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.019949814, 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.019949814 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.771 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.772 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.773 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.774 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-06-02T15:57:16.775 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 18510, 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 114.566614682, 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.006189444 2024-06-02T15:57:16.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 16603, 2024-06-02T15:57:16.778 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 16603, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 822, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:16.779 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:16.780 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:16.781 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:16.782 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:16.783 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:16.783 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:16.783 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:16.783 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-06-02T15:57:16.784 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:16.784 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:16.784 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:16.784 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:16.785 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 291449, 2024-06-02T15:57:16.785 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 122, 2024-06-02T15:57:16.785 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 704, 2024-06-02T15:57:16.785 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 8, 2024-06-02T15:57:16.786 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-06-02T15:57:16.786 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-06-02T15:57:16.786 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:16.786 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:16.787 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:16.787 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:16.787 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:16.787 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 172, 2024-06-02T15:57:16.787 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 102663, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 546, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-06-02T15:57:16.788 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 70396, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 194269579, 2024-06-02T15:57:16.789 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 33, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 70396, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 70396, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 554.636700534, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.007878809 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-06-02T15:57:16.790 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-06-02T15:57:16.791 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 70396, 2024-06-02T15:57:16.791 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 71861 2024-06-02T15:57:16.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.791 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 70396, 2024-06-02T15:57:16.791 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 28063, 2024-06-02T15:57:16.793 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 42333, 2024-06-02T15:57:16.794 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-06-02T15:57:16.794 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-06-02T15:57:16.794 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-06-02T15:57:16.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-06-02T15:57:16.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 312, 2024-06-02T15:57:16.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 2748, 2024-06-02T15:57:16.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 139, 2024-06-02T15:57:16.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 38206, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 27552, 2024-06-02T15:57:16.796 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:16.797 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-06-02T15:57:16.798 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-06-02T15:57:16.799 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 94, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 6, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 12, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 12, 2024-06-02T15:57:16.800 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 2364, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 398, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 112 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-06-02T15:57:16.801 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-06-02T15:57:16.802 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 16, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 8, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 37031, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 18511, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.803 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 232383, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 73878325, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 232383, 2024-06-02T15:57:16.804 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 232383, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 73878325, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.805 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-06-02T15:57:16.806 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 70396, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 196284158, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 70396, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 196284158, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.807 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:16.808 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 70396, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 70396, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 70396, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 70396, 2024-06-02T15:57:16.809 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:16.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 18511, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2017763, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 18511, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:16.811 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 312, 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2017763, 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:16.812 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x563cade8e500": { 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 78486, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 181284189, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 78486, 2024-06-02T15:57:16.813 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 2436, 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 181284189, 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:16.814 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:16.815 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:16.815 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.815 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:16.815 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:57:16.815 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:16 smithi120 ceph-mon[36451]: pgmap v903: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:17.872 INFO:tasks.check_counter:Daemon mds.a mds.exported=315 2024-06-02T15:57:17.872 INFO:tasks.check_counter:Daemon mds.a mds.imported=425 2024-06-02T15:57:17.872 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=112 2024-06-02T15:57:17.872 DEBUG:tasks.check_counter:Getting stats from d 2024-06-02T15:57:17.872 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-02T15:57:18.322 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:18.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:18 smithi090 ceph-mon[31595]: pgmap v904: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:18.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:18 smithi120 ceph-mon[36451]: pgmap v904: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:18.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:18 smithi135 ceph-mon[36839]: pgmap v904: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:20.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:20 smithi090 ceph-mon[31595]: pgmap v905: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:20 smithi120 ceph-mon[36451]: pgmap v905: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:20.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:20 smithi135 ceph-mon[36839]: pgmap v905: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:21.239 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:57:21.239 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:21.239 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 324, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 319, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 545488, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 196192, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.091267281, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.033165375, 2024-06-02T15:57:21.240 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.040724422, 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.013234870, 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 319, 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.035949968, 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000112695 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:21.241 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 545488, 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 196192 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.242 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 243, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 320, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 200240, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.043473845, 2024-06-02T15:57:21.243 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.035468489, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.002976558, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 241, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.029628278, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000122938 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.244 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 200240 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.245 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:21.246 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.001199743, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.001006346, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:21.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:21.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:21.249 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:21.250 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:21.251 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:21.253 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 148704, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 29, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-06-02T15:57:21.254 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:21.255 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-06-02T15:57:21.256 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 322, 2024-06-02T15:57:21.257 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 497304, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 322, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 322, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 497304, 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:21.258 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:21.259 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:21.259 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:21.259 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:21.259 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:21.259 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:57:22.187 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-02T15:57:22.187 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-02T15:57:22.188 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-06-02T15:57:22.188 DEBUG:tasks.check_counter:Getting stats from g 2024-06-02T15:57:22.188 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-02T15:57:22.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:22 smithi090 ceph-mon[31595]: pgmap v906: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:22.632 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:22 smithi120 ceph-mon[36451]: pgmap v906: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:22 smithi135 ceph-mon[36839]: pgmap v906: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:24.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:24 smithi090 ceph-mon[31595]: pgmap v907: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:24 smithi120 ceph-mon[36451]: pgmap v907: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:24 smithi135 ceph-mon[36839]: pgmap v907: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:25.311 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 71076, 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 76975, 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 65918903, 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 94409447, 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 31, 2024-06-02T15:57:25.312 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 13.741534997, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 5.781849311, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 4.452527214, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 3.955957854, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 76973, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 16.252235792, 2024-06-02T15:57:25.313 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000211142 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 87416, 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.032103012, 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000367 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-06-02T15:57:25.314 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 896 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 82289, 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 142827, 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 30352555, 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 136625745, 2024-06-02T15:57:25.315 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 14, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 27.658266345, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 18.952387344, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 19.438382991, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 2.907332054, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 142827, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 40.239373866, 2024-06-02T15:57:25.316 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000281735 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 161442, 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.054730594, 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000339 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 665664, 2024-06-02T15:57:25.317 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 321032 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 29400, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 29239, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 16807241, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 35626907, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 8, 2024-06-02T15:57:25.318 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 6.510778348, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 1.668524571, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 1.837979181, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 3.146373044, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 29237, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 7.742988132, 2024-06-02T15:57:25.319 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000264835 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 25664, 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.007530085, 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000293 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.320 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-02T15:57:25.321 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 476256 2024-06-02T15:57:25.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.321 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-06-02T15:57:25.321 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-06-02T15:57:25.321 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 11829, 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 96.627318804, 2024-06-02T15:57:25.322 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.008168680 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 27191, 2024-06-02T15:57:25.323 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 19.013826695, 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000699269 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 46627, 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 45883, 2024-06-02T15:57:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 45883, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 281.292926268, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.006130656 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 744, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 402, 2024-06-02T15:57:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 864, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 97, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 77, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 33, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 9, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 7, 2024-06-02T15:57:25.326 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 17, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 17, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 24481, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 4, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 74810, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 64226, 2024-06-02T15:57:25.327 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 743, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 5, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 204, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-06-02T15:57:25.328 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 364, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 15481, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 568, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 24726, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 18012, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-06-02T15:57:25.329 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 4, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 234, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 18012, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 3, 2024-06-02T15:57:25.330 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 3, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 18009, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 7, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 1554, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 27173, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:25.331 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 16721, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 15185, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 726, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 35537 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-06-02T15:57:25.332 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 318, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 2, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 17534, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 17532, 2024-06-02T15:57:25.333 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:25.334 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 378, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 175, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.335 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 4, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 73726, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 73726, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-06-02T15:57:25.336 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 93, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 93, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 7, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 143278255, 2024-06-02T15:57:25.337 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 0, 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1797, 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 137.378580398, 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.076448848 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 0 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.338 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-06-02T15:57:25.339 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 35, 2024-06-02T15:57:25.339 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 21546, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 21511, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 103, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 1417, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 1314, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 33, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 49692, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 49659, 2024-06-02T15:57:25.340 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 42461, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 42461, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 160604, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 223608 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 58739, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 6258, 2024-06-02T15:57:25.341 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 46627, 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 14, 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 21208, 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 9070, 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 64.365934227, 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.007096574 2024-06-02T15:57:25.342 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 367, 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.853372997, 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002325267 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.343 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-06-02T15:57:25.344 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 17972, 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 62.925113390, 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.003501286 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.345 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-06-02T15:57:25.346 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.347 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.348 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:25.348 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.348 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.349 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.349 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.350 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-06-02T15:57:25.351 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 940, 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 2.246434978, 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.002389824 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.353 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.354 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2, 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.030537233, 2024-06-02T15:57:25.355 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.015268616 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:25.356 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.357 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:25.358 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.359 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-06-02T15:57:25.360 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 17532, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 150.871533443, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.008605494 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.361 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.362 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 16690, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 16690, 2024-06-02T15:57:25.363 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 2402, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:25.364 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:25.365 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:25.366 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:25.370 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:25.370 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:25.370 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:25.371 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 301326, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 144, 2024-06-02T15:57:25.372 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 1672, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 19, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:25.373 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 172, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 252348, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2133, 2024-06-02T15:57:25.374 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-06-02T15:57:25.375 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 58524, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 149406712, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 27, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 58524, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 58524, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 517.342258844, 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.008839830 2024-06-02T15:57:25.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 58524, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 60095 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 58524, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 18826, 2024-06-02T15:57:25.377 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 39698, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 0, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 10, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 412, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 2209, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 161, 2024-06-02T15:57:25.378 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 36187, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-06-02T15:57:25.379 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 18012, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 20, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-06-02T15:57:25.380 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-06-02T15:57:25.381 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-06-02T15:57:25.382 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 94, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 6, 2024-06-02T15:57:25.383 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 12, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 12, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 0, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 2173, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 804, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 107 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.384 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 1, 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.385 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 16, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 8, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 35064, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 17532, 2024-06-02T15:57:25.386 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 182761, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 96607360, 2024-06-02T15:57:25.387 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 182761, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 182761, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 96607360, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.388 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-06-02T15:57:25.389 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 58524, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 151311268, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 58524, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 151311268, 2024-06-02T15:57:25.390 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:25.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 58524, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 58524, 2024-06-02T15:57:25.392 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 58524, 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 58524, 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.393 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 17532, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 1910988, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 17532, 2024-06-02T15:57:25.394 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 412, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 1910988, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.395 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55da261c8500": { 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 73726, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 139083951, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:25.396 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 73726, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 1797, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 139083951, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:25.397 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:25.398 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.398 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:25.398 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:57:26.360 INFO:tasks.check_counter:Daemon mds.g mds.exported=364 2024-06-02T15:57:26.360 INFO:tasks.check_counter:Daemon mds.g mds.imported=568 2024-06-02T15:57:26.360 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=97 2024-06-02T15:57:26.360 DEBUG:tasks.check_counter:Getting stats from j 2024-06-02T15:57:26.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-02T15:57:26.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:26 smithi090 ceph-mon[31595]: pgmap v908: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:26.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:26 smithi120 ceph-mon[36451]: pgmap v908: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:26.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:26 smithi135 ceph-mon[36839]: pgmap v908: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:28.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:28 smithi090 ceph-mon[31595]: pgmap v909: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:57:28.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:28 smithi120 ceph-mon[36451]: pgmap v909: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:57:28.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:28 smithi135 ceph-mon[36839]: pgmap v909: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s 2024-06-02T15:57:29.188 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-06-02T15:57:29.188 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.004001157, 2024-06-02T15:57:29.189 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.001159547, 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:29.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:29.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 6, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 248, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3264, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 201216, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.046620126, 2024-06-02T15:57:29.192 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.036330514, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.004830184, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 247, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.029702804, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000120254 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.193 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 201216 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.194 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 324, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 321, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 543632, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 198304, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.109646809, 2024-06-02T15:57:29.195 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.050431071, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.046270534, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.013414582, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 320, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.040742139, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000127319 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.196 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 543632, 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 198304 2024-06-02T15:57:29.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.198 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-06-02T15:57:29.198 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-06-02T15:57:29.198 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-06-02T15:57:29.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.198 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:29.199 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:29.200 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:29.200 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:29.200 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:29.200 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:29.201 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:29.202 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:29.203 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:29.203 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:29.203 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:29.203 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:29.203 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:29.204 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 150766, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 31, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-06-02T15:57:29.205 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:29.206 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:29.207 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 326, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 497824, 2024-06-02T15:57:29.208 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 326, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 326, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 497824, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-06-02T15:57:29.209 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-06-02T15:57:29.210 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-06-02T15:57:29.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:29.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-06-02T15:57:29.210 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-06-02T15:57:30.027 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-02T15:57:30.028 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-02T15:57:30.028 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-06-02T15:57:30.028 DEBUG:tasks.check_counter:Getting stats from b 2024-06-02T15:57:30.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-02T15:57:30.275 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:57:30.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:30 smithi090 ceph-mon[31595]: pgmap v910: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:57:30.711 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:30 smithi120 ceph-mon[36451]: pgmap v910: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:57:30.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:30 smithi135 ceph-mon[36839]: pgmap v910: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 8 op/s 2024-06-02T15:57:32.194 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-02T15:57:32.194 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 47370, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 58005, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 57319897, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 118468960, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 135, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 6, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 4.019538313, 2024-06-02T15:57:32.195 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 1.286085671, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 1.080434767, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 1.408332209, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 58007, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 21.569436360, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000371841 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.196 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 31754, 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.005240178, 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000165 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.197 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 48192, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 48557, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 23239939, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 94541850, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 37, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 3.346851758, 2024-06-02T15:57:32.198 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.960702063, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.967503592, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 1.415536433, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 48559, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 23.424671550, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000482396 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.199 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 25709, 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.003953479, 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000153 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 684816, 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 320704 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:32.200 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 119606, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 202535, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 44727951, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 136481938, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 19, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 8, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 8.256326378, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 3.238288616, 2024-06-02T15:57:32.201 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 4.342410966, 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 1.478302663, 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 202527, 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 28.314936719, 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000139808 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.202 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 197083, 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.033165912, 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000168 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 64560, 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 616112 2024-06-02T15:57:32.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-02T15:57:32.204 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 22287, 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 53.551950099, 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002402833 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-02T15:57:32.205 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 26956, 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 4.014826921, 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000148940 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-02T15:57:32.206 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 61643, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 50635, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 50635, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 197.668703823, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003903795 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-02T15:57:32.207 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 11008, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 559, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 4, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 8215, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 156, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 29, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 26, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 4, 2024-06-02T15:57:32.208 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 7, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 15, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 19, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 30430, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 4, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 4, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 4, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 83866, 2024-06-02T15:57:32.209 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 64792, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 11003, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 109, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-02T15:57:32.210 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 526, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 20789, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 212, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 19971, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 19310, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-02T15:57:32.211 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 4, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 234, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 19310, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 106, 2024-06-02T15:57:32.212 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 11, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 95, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 19204, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 7, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 156, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 35917, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 1, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:32.213 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 16815, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 16651, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 400, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 28882 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-06-02T15:57:32.214 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 164, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 1, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:32.215 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 14096, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 14095, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:32.216 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 11, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 542, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 189, 2024-06-02T15:57:32.217 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 1, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 77903, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 77787, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 77787, 2024-06-02T15:57:32.218 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 116, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 108, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 19, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 127, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 125, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 125, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 2, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 2, 2024-06-02T15:57:32.219 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 1, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 1, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 157349005, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 157601007, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 0, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2281, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 116.650137847, 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.051139911 2024-06-02T15:57:32.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.221 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 0 2024-06-02T15:57:32.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.221 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-02T15:57:32.222 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 30, 2024-06-02T15:57:32.222 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 28372, 2024-06-02T15:57:32.222 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 28342, 2024-06-02T15:57:32.222 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 25, 2024-06-02T15:57:32.222 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 3152, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 3127, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 26, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 56812, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 56786, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 4, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 46617, 2024-06-02T15:57:32.223 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 46613, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 168340, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 231800 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 70446, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 1763, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 61643, 2024-06-02T15:57:32.224 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 124, 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 11397, 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 21155, 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 108.806264285, 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.005143288 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-02T15:57:32.225 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 384, 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.246089153, 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000640857 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 13978, 2024-06-02T15:57:32.227 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 13.754106669, 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000983982 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.228 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000225836, 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000225836 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.229 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.230 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:32.231 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.231 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.231 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.232 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 130, 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.126457607, 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000972750 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.233 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-02T15:57:32.234 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1, 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000302075, 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000302075 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 830, 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 1.568360481, 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001889590 2024-06-02T15:57:32.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 4, 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.008638518, 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002159629 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.236 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.237 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 118, 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.378256493, 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003205563 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.238 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.011418581, 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003806193 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.239 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-02T15:57:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5, 2024-06-02T15:57:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.009177343, 2024-06-02T15:57:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.001835468 2024-06-02T15:57:32.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.241 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:32.242 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 22, 2024-06-02T15:57:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 10.025634680, 2024-06-02T15:57:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.455710667 2024-06-02T15:57:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.243 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-02T15:57:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-02T15:57:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 25, 2024-06-02T15:57:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.086822085, 2024-06-02T15:57:32.245 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.003472883 2024-06-02T15:57:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-02T15:57:32.246 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-02T15:57:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 13977, 2024-06-02T15:57:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 62.646217957, 2024-06-02T15:57:32.248 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.004482093 2024-06-02T15:57:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:32.249 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 2, 2024-06-02T15:57:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000732060, 2024-06-02T15:57:32.250 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000366030 2024-06-02T15:57:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-02T15:57:32.251 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-02T15:57:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 3, 2024-06-02T15:57:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 2, 2024-06-02T15:57:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-02T15:57:32.252 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-02T15:57:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 1384, 2024-06-02T15:57:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 1384, 2024-06-02T15:57:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 857, 2024-06-02T15:57:32.253 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-02T15:57:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:32.255 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:32.256 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:32.257 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:32.258 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:32.259 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:32.260 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 290911, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 119, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 440, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 5, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-02T15:57:32.261 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:32.262 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 124464, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 926, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:32.263 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 108298, 2024-06-02T15:57:32.264 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 211195772, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 37, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 108298, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 108298, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 41512.842574797, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.383320491 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.265 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 108300, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 193166 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 108300, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 20247, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 88053, 2024-06-02T15:57:32.266 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 6897, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 46903, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 363, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 2633, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 328, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-02T15:57:32.267 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 5, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 29552, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 19336, 2024-06-02T15:57:32.268 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 70746, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:32.269 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 1, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-02T15:57:32.270 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-02T15:57:32.271 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 94, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 15, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 12, 2024-06-02T15:57:32.272 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 12, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 14977, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 1126, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 299 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.273 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 1, 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-02T15:57:32.274 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 80, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 9, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 28164, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 14095, 2024-06-02T15:57:32.275 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 215164, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 105891225, 2024-06-02T15:57:32.276 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 215164, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 215164, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 105891225, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.277 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:32.278 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 108300, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 212722550, 2024-06-02T15:57:32.279 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 108300, 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 212722550, 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.280 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:32.281 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 108300, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 108300, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 108300, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 108300, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.282 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 14095, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 1535581, 2024-06-02T15:57:32.283 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 14095, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 352, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 1535581, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.284 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55bf19bb0500": { 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-02T15:57:32.285 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 77903, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 153406703, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 77903, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 2281, 2024-06-02T15:57:32.286 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 153406703, 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:32.287 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-02T15:57:32.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:32 smithi090 ceph-mon[31595]: pgmap v911: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:32.635 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:32 smithi120 ceph-mon[36451]: pgmap v911: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:32.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:32 smithi135 ceph-mon[36839]: pgmap v911: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:32.990 INFO:tasks.check_counter:Daemon mds.b mds.exported=526 2024-06-02T15:57:32.990 INFO:tasks.check_counter:Daemon mds.b mds.imported=212 2024-06-02T15:57:32.990 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=156 2024-06-02T15:57:32.990 DEBUG:tasks.check_counter:Getting stats from e 2024-06-02T15:57:32.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-02T15:57:33.235 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:57:34.620 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:34 smithi120 ceph-mon[36451]: pgmap v912: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:34.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:34 smithi090 ceph-mon[31595]: pgmap v912: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:34.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:34 smithi135 ceph-mon[36839]: pgmap v912: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:35.053 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-02T15:57:35.054 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:35.054 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 750, 2024-06-02T15:57:35.054 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 724, 2024-06-02T15:57:35.054 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 26597966, 2024-06-02T15:57:35.054 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 324360, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.101242818, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.028893361, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.047969384, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.019017945, 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:35.055 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 724, 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.031841260, 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000043979 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.056 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 655840, 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 200016 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1119, 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1362, 2024-06-02T15:57:35.057 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 71595248, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 923419, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 7, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.155714299, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.038144626, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.071532976, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.034138419, 2024-06-02T15:57:35.058 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1360, 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.053943289, 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000039664 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.059 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 568208 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1917, 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 1919, 2024-06-02T15:57:35.060 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 42532615, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 609664, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 6, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.208554794, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.063746152, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.069385964, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.057968858, 2024-06-02T15:57:35.061 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 1917, 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.081379708, 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000042451 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.062 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 1, 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-02T15:57:35.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-MDSRank": { 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 5736, 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.154319846, 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000026903 2024-06-02T15:57:35.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "queue_len": 0, 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "complete_latency": { 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "mds": { 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "request": 0, 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "reply": 0, 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "reply_latency": { 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.066 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_reply": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "forward": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_complete": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_fetch_keys": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_commit": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_split": 0, 2024-06-02T15:57:35.067 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_merge": 0, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes": 37467, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_top": 5624, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_bottom": 31843, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pin_tail": 0, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_pinned": 29432, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_expired": 0, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "inodes_with_caps": 0, 2024-06-02T15:57:35.068 INFO:teuthology.orchestra.run.smithi120.stdout: "caps": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "subtrees": 1, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_hit": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_forward": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_discover": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_dir_fetch": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:35.069 INFO:teuthology.orchestra.run.smithi120.stdout: "traverse_lock": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "load_cent": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "q": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "exported": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "exported_inodes": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "imported": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "imported_inodes": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:35.070 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_backtrace_fetch": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "openino_peer_discover": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rfiles": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rbytes": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "root_rsnaps": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:35.071 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_inodes": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "scrub_file_inodes": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_inode_file_caps": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_grant": 0, 2024-06-02T15:57:35.072 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_caps_dirty": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_cap_release": 0, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "process_request_cap_release": 0 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.073 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_cache": { 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_try_discover": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_send_discover": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "dir_handle_discover": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:35.074 INFO:teuthology.orchestra.run.smithi120.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_created": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_enqueued": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "strays_migrated": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:35.075 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_started": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "recovery_completed": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_exportdir": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_flush": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragmentdir": 0, 2024-06-02T15:57:35.076 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "ireq_inodestats": 0 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_log": { 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "evlrg": 0, 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "evadd": 0, 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "evex": 0, 2024-06-02T15:57:35.077 INFO:teuthology.orchestra.run.smithi120.stdout: "evtrm": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "ev": 73726, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "evexg": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "evexd": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "segadd": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "segex": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "segtrm": 0, 2024-06-02T15:57:35.078 INFO:teuthology.orchestra.run.smithi120.stdout: "seg": 93, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "segmjr": 7, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "segexg": 0, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "segexd": 0, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "expos": 4194304, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "wrpos": 143278255, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "rdpos": 143277117, 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "jlat": { 2024-06-02T15:57:35.079 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: "replayed": 73726 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_mem": { 2024-06-02T15:57:35.080 INFO:teuthology.orchestra.run.smithi120.stdout: "ino": 15, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "ino+": 20371, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "ino-": 20356, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dir": 83, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dir+": 1406, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dir-": 1323, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dn": 37467, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dn+": 53013, 2024-06-02T15:57:35.081 INFO:teuthology.orchestra.run.smithi120.stdout: "dn-": 15546, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "cap": 0, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "cap+": 0, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "cap-": 0, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "rss": 112396, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "heap": 215416 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_server": { 2024-06-02T15:57:35.082 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_client_request": 0, 2024-06-02T15:57:35.083 INFO:teuthology.orchestra.run.smithi120.stdout: "dispatch_server_request": 0, 2024-06-02T15:57:35.083 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_request": 0, 2024-06-02T15:57:35.083 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_client_session": 0, 2024-06-02T15:57:35.083 INFO:teuthology.orchestra.run.smithi120.stdout: "handle_peer_request": 0, 2024-06-02T15:57:35.083 INFO:teuthology.orchestra.run.smithi120.stdout: "req_create_latency": { 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getattr_latency": { 2024-06-02T15:57:35.084 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.085 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "req_link_latency": { 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookup_latency": { 2024-06-02T15:57:35.086 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupino_latency": { 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupname_latency": { 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.088 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.089 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "req_lssnap_latency": { 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.090 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.091 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mkdir_latency": { 2024-06-02T15:57:35.091 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mknod_latency": { 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "req_mksnap_latency": { 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "req_open_latency": { 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.093 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.094 INFO:teuthology.orchestra.run.smithi120.stdout: "req_readdir_latency": { 2024-06-02T15:57:35.094 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.094 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rename_latency": { 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.095 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:35.096 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmdir_latency": { 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:35.098 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setattr_latency": { 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.100 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setlayout_latency": { 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.101 INFO:teuthology.orchestra.run.smithi120.stdout: "req_setxattr_latency": { 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "req_symlink_latency": { 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.102 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: "req_unlink_latency": { 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.103 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_sessions": { 2024-06-02T15:57:35.104 INFO:teuthology.orchestra.run.smithi120.stdout: "session_count": 1, 2024-06-02T15:57:35.105 INFO:teuthology.orchestra.run.smithi120.stdout: "session_add": 2, 2024-06-02T15:57:35.105 INFO:teuthology.orchestra.run.smithi120.stdout: "session_remove": 1, 2024-06-02T15:57:35.105 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_open": 1, 2024-06-02T15:57:35.105 INFO:teuthology.orchestra.run.smithi120.stdout: "sessions_stale": 0, 2024-06-02T15:57:35.105 INFO:teuthology.orchestra.run.smithi120.stdout: "total_load": 0, 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: "average_load": 0, 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: "avg_session_uptime": 2745, 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-02T15:57:35.106 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:35.107 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:35.107 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:35.108 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:35.109 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:35.110 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:35.111 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:35.111 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:35.111 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:35.111 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:35.111 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:35.112 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 224131, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 74, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-02T15:57:35.113 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:35.114 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 172, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 26353589, 2024-06-02T15:57:35.115 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 128912, 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.116 INFO:teuthology.orchestra.run.smithi120.stdout: "objecter": { 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_active": 0, 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_laggy": 0, 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send": 3425, 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_send_bytes": 21, 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_resend": 3, 2024-06-02T15:57:35.117 INFO:teuthology.orchestra.run.smithi120.stdout: "op_reply": 3425, 2024-06-02T15:57:35.120 INFO:teuthology.orchestra.run.smithi120.stdout: "op_latency": { 2024-06-02T15:57:35.120 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3425, 2024-06-02T15:57:35.120 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 7.718825323, 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.002253671 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "op_inflight": 0, 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "oplen_avg": { 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3425, 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 3426 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.121 INFO:teuthology.orchestra.run.smithi120.stdout: "op": 3425, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "op_r": 3425, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "op_w": 0, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "op_rmw": 0, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "op_pg": 0, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_stat": 2107, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_create": 0, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_read": 1316, 2024-06-02T15:57:35.122 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_write": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writefull": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_writesame": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_append": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_zero": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_truncate": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_delete": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_mapext": 0, 2024-06-02T15:57:35.123 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_getxattr": 1, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_setxattr": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_call": 0, 2024-06-02T15:57:35.124 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_watch": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_notify": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "osdop_other": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_active": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_send": 0, 2024-06-02T15:57:35.125 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_resend": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "linger_ping": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_active": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_send": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolop_resend": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_active": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_send": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "poolstat_resend": 0, 2024-06-02T15:57:35.126 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_active": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_send": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "statfs_resend": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "command_active": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "command_send": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "command_resend": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "map_epoch": 94, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "map_full": 0, 2024-06-02T15:57:35.127 INFO:teuthology.orchestra.run.smithi120.stdout: "map_inc": 6, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_sessions": 9, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_open": 9, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_session_close": 0, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_laggy": 0, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_wr": 0, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_rd": 2, 2024-06-02T15:57:35.128 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_del": 0 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: "oft": { 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_objs": 0, 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_updates": 0, 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: "omap_total_removes": 0 2024-06-02T15:57:35.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.130 INFO:teuthology.orchestra.run.smithi120.stdout: "osd-slow-ops": { 2024-06-02T15:57:35.130 INFO:teuthology.orchestra.run.smithi120.stdout: "slow_ops_count": 0 2024-06-02T15:57:35.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.131 INFO:teuthology.orchestra.run.smithi120.stdout: "purge_queue": { 2024-06-02T15:57:35.131 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:35.131 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T15:57:35.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing": 0, 2024-06-02T15:57:35.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executing_high_water": 0, 2024-06-02T15:57:35.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed_ops": 0, 2024-06-02T15:57:35.132 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_executed": 0, 2024-06-02T15:57:35.133 INFO:teuthology.orchestra.run.smithi120.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:35.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.133 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:35.133 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:35.134 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:35.134 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:35.134 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 3782, 2024-06-02T15:57:35.134 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 140364289, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 3782, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3782, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 140364289, 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:35.135 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:35.136 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3425, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 139105817, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3425, 2024-06-02T15:57:35.137 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 139105817, 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.138 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 1024, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:35.139 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 3425, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 3425, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 3425, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 3425, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-02T15:57:35.142 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-write_buf_throttle-0x55dd72e366e0": { 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 3758096384, 2024-06-02T15:57:35.143 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 0, 2024-06-02T15:57:35.144 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 0, 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:35.145 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-02T15:57:35.939 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-02T15:57:35.940 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-02T15:57:35.940 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-06-02T15:57:35.940 DEBUG:tasks.check_counter:Getting stats from h 2024-06-02T15:57:35.940 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-02T15:57:36.178 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:57:36.556 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:36 smithi120 ceph-mon[36451]: pgmap v913: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:36.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:36 smithi090 ceph-mon[31595]: pgmap v913: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:36.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:36 smithi135 ceph-mon[36839]: pgmap v913: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:37.983 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-02T15:57:37.983 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 4, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 4, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 2944, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 448, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.002002506, 2024-06-02T15:57:37.984 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000048912, 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000719052, 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 3, 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000871859, 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000290619 2024-06-02T15:57:37.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T15:57:37.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 119, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 118, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 185904, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 71728, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:37.987 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.016369224, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.006212797, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.006224385, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.002805479, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 117, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.005343179, 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000045668 2024-06-02T15:57:37.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 185904, 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 71728 2024-06-02T15:57:37.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 1, 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 89, 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 160, 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 94224, 2024-06-02T15:57:37.990 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 3, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.004903802, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.003509207, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000473361, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:37.991 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 87, 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.004916262, 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000056508 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:37.992 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 94224 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-02T15:57:37.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:37.994 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:37.995 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:37.996 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:37.997 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148801, 2024-06-02T15:57:37.998 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 27, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:37.999 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:38.000 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:38.001 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 120, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 169106, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 120, 2024-06-02T15:57:38.002 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 120, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 169106, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:38.003 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:38.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:38.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:38.004 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-02T15:57:38.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:38 smithi090 ceph-mon[31595]: pgmap v914: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:38.780 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:38 smithi120 ceph-mon[36451]: pgmap v914: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:38.808 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-06-02T15:57:38.808 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-06-02T15:57:38.809 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.h 2024-06-02T15:57:38.809 DEBUG:tasks.check_counter:Getting stats from k 2024-06-02T15:57:38.809 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-02T15:57:38.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:38 smithi135 ceph-mon[36839]: pgmap v914: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:39.046 INFO:teuthology.orchestra.run.smithi120.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T15:57:40.537 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:40 smithi120 ceph-mon[36451]: pgmap v915: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:40.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:40 smithi090 ceph-mon[31595]: pgmap v915: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:40.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:40 smithi135 ceph-mon[36839]: pgmap v915: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:40.819 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-06-02T15:57:40.819 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:40.819 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 328, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 325, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 544720, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 200880, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.049967500, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.019494399, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.019874821, 2024-06-02T15:57:40.820 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.008085535, 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 324, 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.014458747, 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000044625 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:40.821 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 544720, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 200880 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 6, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 251, 2024-06-02T15:57:40.822 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 3264, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 201744, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.014540391, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.010721825, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.001152604, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:40.823 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 250, 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.016734575, 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000066938 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:40.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 201744 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_messages": 0, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages": 2, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_bytes": 0, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_bytes": 0, 2024-06-02T15:57:40.825 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_total_time": 0.000437797, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_recv_time": 0.000366986, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:40.826 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:40.827 INFO:teuthology.orchestra.run.smithi120.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "cct": { 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "total_workers": 0, 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "unhealthy_workers": 0 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "mempool": { 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:40.828 INFO:teuthology.orchestra.run.smithi120.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:40.829 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:40.830 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:40.831 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_items": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_bytes": 148784, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_anon_items": 31, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_bytes": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_items": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_items": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:40.834 INFO:teuthology.orchestra.run.smithi120.stdout: "pgmap_items": 0, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "mds_co_items": 2, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_1_items": 0, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "unittest_2_items": 0 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-06-02T15:57:40.835 INFO:teuthology.orchestra.run.smithi120.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "val": 0, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "max": 104857600, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "get_started": 0, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "get": 330, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "get_sum": 498344, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "get_or_fail_success": 330, 2024-06-02T15:57:40.836 INFO:teuthology.orchestra.run.smithi120.stdout: "take": 0, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "take_sum": 0, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "put": 330, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "put_sum": 498344, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "wait": { 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "avgcount": 0, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "sum": 0.000000000, 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: "avgtime": 0.000000000 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:40.837 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-06-02T15:57:40.838 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-06-02T15:57:41.538 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-06-02T15:57:41.538 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-06-02T15:57:41.539 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-06-02T15:57:41.539 DEBUG:tasks.check_counter:Getting stats from c 2024-06-02T15:57:41.539 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-02T15:57:41.787 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:57:42.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:42 smithi135 ceph-mon[36839]: pgmap v916: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:42.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:42 smithi090 ceph-mon[31595]: pgmap v916: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:42.770 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:42 smithi120 ceph-mon[36451]: pgmap v916: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:43.795 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 943, 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 927, 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 30160008, 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 389360, 2024-06-02T15:57:43.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.129740446, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.037792466, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.054988850, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.028385380, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 927, 2024-06-02T15:57:43.797 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.045268558, 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048833 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 611808, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 200464 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 548, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 792, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 44511715, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 524467, 2024-06-02T15:57:43.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.081904994, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.026569091, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.033515584, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.014186421, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 790, 2024-06-02T15:57:43.800 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.039814213, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000050397 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-02T15:57:43.801 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 350464 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 369, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 371, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 15651462, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 117358, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-06-02T15:57:43.802 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.046525358, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.010182555, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.018281730, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.014524777, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 369, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.024420390, 2024-06-02T15:57:43.803 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000066179 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:43.804 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-02T15:57:43.805 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2490, 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.228874528, 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000091917 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.806 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:43.809 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-02T15:57:43.809 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-02T15:57:43.809 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.809 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.809 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.810 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-06-02T15:57:43.811 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 14002, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 9772, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 4230, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 41, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-06-02T15:57:43.812 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:43.813 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:43.814 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:43.815 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-06-02T15:57:43.816 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-02T15:57:43.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:43.818 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-02T15:57:43.819 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-02T15:57:43.820 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 46829, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-02T15:57:43.821 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 18, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-06-02T15:57:43.822 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 157349005, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 157601007, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 157599559, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.823 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 46829 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 20, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 16193, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 16173, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 19, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 1217, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 1198, 2024-06-02T15:57:43.824 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 14002, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 44722, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 30720, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 81168, 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215416 2024-06-02T15:57:43.825 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-02T15:57:43.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.828 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:43.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.830 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.831 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.832 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-02T15:57:43.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.833 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.833 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.833 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.834 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-02T15:57:43.834 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.834 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.834 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.834 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.835 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.836 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.838 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:43.839 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.840 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.841 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-02T15:57:43.842 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.843 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.844 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.845 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 0, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-06-02T15:57:43.846 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 2753, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:43.847 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:43.848 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:43.849 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:43.850 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:43.851 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 231424, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 76, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:43.852 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 172, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:43.853 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 9927547, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 56363, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-02T15:57:43.854 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 1510, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 26, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 4, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 1510, 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-02T15:57:43.855 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1510, 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3.464924792, 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002294652 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1510, 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1511 2024-06-02T15:57:43.856 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 1510, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 1510, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 904, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-06-02T15:57:43.857 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 603, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-06-02T15:57:43.858 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 2, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:43.859 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-02T15:57:43.860 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-02T15:57:43.861 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-02T15:57:43.862 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-02T15:57:43.862 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-02T15:57:43.862 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-02T15:57:43.862 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-02T15:57:43.862 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 94, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 4, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 10, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 10, 2024-06-02T15:57:43.863 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-06-02T15:57:43.864 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-02T15:57:43.864 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-06-02T15:57:43.865 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-06-02T15:57:43.865 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-06-02T15:57:43.865 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.865 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-02T15:57:43.866 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-06-02T15:57:43.867 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 1856, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 90135492, 2024-06-02T15:57:43.868 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 1856, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1856, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 90135492, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.869 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:43.870 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1510, 2024-06-02T15:57:43.871 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 89283679, 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1510, 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 89283679, 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.872 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:43.873 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1510, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 1510, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1510, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1510, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.874 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:43.875 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-02T15:57:43.876 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x564e2956c6e0": { 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:43.877 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:43.878 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:43.879 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-02T15:57:44.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:44 smithi135 ceph-mon[36839]: pgmap v917: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:44.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:44 smithi090 ceph-mon[31595]: pgmap v917: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:44.684 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-06-02T15:57:44.684 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-06-02T15:57:44.684 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-06-02T15:57:44.684 DEBUG:tasks.check_counter:Getting stats from f 2024-06-02T15:57:44.685 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-02T15:57:44.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:44 smithi120 ceph-mon[36451]: pgmap v917: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 9 op/s 2024-06-02T15:57:44.944 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:57:45.783 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:45 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:57:45.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:45 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:57:45.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:45 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T15:57:46.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.664 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:57:46.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:57:46.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:57:46.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:57:46.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:57:46.665 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:46 smithi135 ceph-mon[36839]: pgmap v918: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:46.665 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 0, 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2, 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 0, 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 0, 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:46.666 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.000922699, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000105884, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:46.667 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:46.668 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 247, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 320, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 200944, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:46.669 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.015785001, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.011752776, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000995725, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 245, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.018285947, 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000074636 2024-06-02T15:57:46.670 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 200944 2024-06-02T15:57:46.671 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 329, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 324, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 546848, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 199408, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:46.672 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.052617800, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.021638870, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.019397177, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.008369527, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 324, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.017498838, 2024-06-02T15:57:46.673 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000054008 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 546848, 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 199408 2024-06-02T15:57:46.674 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:46.675 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:46.676 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:46.677 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:46.678 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:46.679 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148699, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 30, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-02T15:57:46.680 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:46.681 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:46.682 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 327, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 497954, 2024-06-02T15:57:46.683 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 327, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 327, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 497954, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:46.684 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:46.685 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:46.685 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:46.685 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:46.685 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-02T15:57:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:57:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:57:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:57:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:57:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:57:46.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:46 smithi120 ceph-mon[36451]: pgmap v918: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-06-02T15:57:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T15:57:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T15:57:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' 2024-06-02T15:57:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:46 smithi090 ceph-mon[31595]: pgmap v918: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 9 op/s 2024-06-02T15:57:47.394 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-06-02T15:57:47.394 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-06-02T15:57:47.395 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-06-02T15:57:47.395 DEBUG:tasks.check_counter:Getting stats from i 2024-06-02T15:57:47.395 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-02T15:57:47.630 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:57:48.776 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:48 smithi135 ceph-mon[36839]: pgmap v919: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:48.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:48 smithi120 ceph-mon[36451]: pgmap v919: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:48.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:48 smithi090 ceph-mon[31595]: pgmap v919: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 8 op/s 2024-06-02T15:57:49.349 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-02T15:57:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:49.349 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1850, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1826, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 34683640, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 675300, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.238620154, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.071997873, 2024-06-02T15:57:49.350 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.084113447, 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.064058362, 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1825, 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.097140851, 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000053227 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:49.351 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 653168, 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 200208 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:49.352 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 798, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1042, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 64057477, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 826422, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.117531605, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.031816254, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.055789256, 2024-06-02T15:57:49.353 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.021029544, 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1040, 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.043785292, 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000042101 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.354 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 573232 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1479, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1479, 2024-06-02T15:57:49.355 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 84536751, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 469505, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.152991051, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.035072245, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.074329757, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.035580185, 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:49.356 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1478, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.052417601, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000035465 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.357 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-06-02T15:57:49.358 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 6114, 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.175347540, 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000028679 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-06-02T15:57:49.359 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-06-02T15:57:49.360 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-06-02T15:57:49.361 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 18521, 2024-06-02T15:57:49.362 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 18521, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 18521, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 1, 2024-06-02T15:57:49.363 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-06-02T15:57:49.364 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-06-02T15:57:49.365 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-06-02T15:57:49.366 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T15:57:49.367 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-06-02T15:57:49.368 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-06-02T15:57:49.369 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-06-02T15:57:49.370 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-06-02T15:57:49.371 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 78486, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-06-02T15:57:49.372 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 99, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 8, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-06-02T15:57:49.373 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 185478493, 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 185477355, 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 78486 2024-06-02T15:57:49.374 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 13, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 30088, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 30075, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 11, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 2054, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 2043, 2024-06-02T15:57:49.375 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 18521, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 60563, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 42042, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 97816, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215416 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.376 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.377 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.378 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.379 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.380 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.381 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.382 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-06-02T15:57:49.383 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.384 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-06-02T15:57:49.385 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.386 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.387 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-06-02T15:57:49.388 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.389 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-06-02T15:57:49.390 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.391 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.392 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-06-02T15:57:49.393 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.394 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.395 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.396 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-06-02T15:57:49.397 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 2759, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:49.399 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:49.400 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:49.401 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:49.402 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 240199, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 79, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-02T15:57:49.403 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 8984, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 172, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:49.404 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 13082376, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 74212, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.405 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 3765, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 21, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 3, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 3765, 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-06-02T15:57:49.406 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3765, 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 7.633834218, 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002027578 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3765, 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3766 2024-06-02T15:57:49.407 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 3765, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 3765, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 2426, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-06-02T15:57:49.408 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1337, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-06-02T15:57:49.409 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 1, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-06-02T15:57:49.410 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-06-02T15:57:49.411 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-06-02T15:57:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-06-02T15:57:49.413 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-06-02T15:57:49.414 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-06-02T15:57:49.415 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 94, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 6, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 11, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 11, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-06-02T15:57:49.416 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-06-02T15:57:49.417 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-06-02T15:57:49.418 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:49.419 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 4123, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 182885586, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 4123, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4123, 2024-06-02T15:57:49.420 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 182885586, 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:49.421 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3765, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 181567494, 2024-06-02T15:57:49.422 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3765, 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 181567494, 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-06-02T15:57:49.423 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3765, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 3765, 2024-06-02T15:57:49.424 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3765, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 3765, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:49.425 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:49.426 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55bb3e2ee6e0": { 2024-06-02T15:57:49.427 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-06-02T15:57:49.428 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:49.429 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-02T15:57:50.105 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-06-02T15:57:50.105 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-06-02T15:57:50.105 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-06-02T15:57:50.105 DEBUG:tasks.check_counter:Getting stats from l 2024-06-02T15:57:50.105 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-02T15:57:50.339 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T15:57:50.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:50 smithi090 ceph-mon[31595]: pgmap v920: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:50.673 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:50 smithi135 ceph-mon[36839]: pgmap v920: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:50.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:50 smithi120 ceph-mon[36451]: pgmap v920: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 8 op/s 2024-06-02T15:57:51.983 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 0, 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2, 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 0, 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 0, 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:51.984 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 0, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.001058512, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000292873, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:51.985 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 336, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 331, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 548752, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 203888, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 2, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.052076705, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.021003264, 2024-06-02T15:57:51.987 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.019185288, 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.008312834, 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 331, 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.020364694, 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000061524 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:51.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 548752, 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 203888 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.989 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 252, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 320, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 201824, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 2, 2024-06-02T15:57:51.990 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.014974746, 2024-06-02T15:57:51.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.011166113, 2024-06-02T15:57:51.993 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.000875955, 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 250, 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.016687308, 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000066749 2024-06-02T15:57:51.994 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T15:57:51.995 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 201824 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 0, 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-06-02T15:57:51.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T15:57:51.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-06-02T15:57:51.998 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T15:57:51.999 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-06-02T15:57:52.000 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 148784, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 31, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-06-02T15:57:52.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 0, 2024-06-02T15:57:52.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 928, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 2, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-06-02T15:57:52.003 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-06-02T15:57:52.004 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-06-02T15:57:52.004 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-06-02T15:57:52.004 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-06-02T15:57:52.004 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T15:57:52.004 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 334, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 498864, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 334, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-06-02T15:57:52.005 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 334, 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 498864, 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:52.006 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-06-02T15:57:52.007 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-06-02T15:57:52.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:52 smithi090 ceph-mon[31595]: pgmap v921: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:52.705 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:52 smithi135 ceph-mon[36839]: pgmap v921: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:52.732 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-02T15:57:52.732 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-02T15:57:52.732 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-06-02T15:57:52.733 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-06-02T15:57:52.743 INFO:tasks.kclient:Unmounting kernel clients... 2024-06-02T15:57:52.743 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:57:52.743 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T15:57:52.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:52 smithi120 ceph-mon[36451]: pgmap v921: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:52.815 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-06-02T15:57:52.815 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T15:57:52.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-02T15:57:53.321 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:54.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:54 smithi090 ceph-mon[31595]: pgmap v922: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:54.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:54 smithi120 ceph-mon[36451]: pgmap v922: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:54.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:54 smithi135 ceph-mon[36839]: pgmap v922: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 9 op/s 2024-06-02T15:57:55.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 -- 172.21.15.90:0/2172260737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c10fd20 msgr2=0x7fef9c112110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:55.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 --2- 172.21.15.90:0/2172260737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c10fd20 0x7fef9c112110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fef9400b0a0 tx=0x7fef9400b3b0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 -- 172.21.15.90:0/2172260737 shutdown_connections 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 --2- 172.21.15.90:0/2172260737 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c10fd20 0x7fef9c112110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 --2- 172.21.15.90:0/2172260737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c075860 0x7fef9c075c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 --2- 172.21.15.90:0/2172260737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c076180 0x7fef9c10f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 -- 172.21.15.90:0/2172260737 >> 172.21.15.90:0/2172260737 conn(0x7fef9c0fef90 msgr2=0x7fef9c1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 -- 172.21.15.90:0/2172260737 shutdown_connections 2024-06-02T15:57:55.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.658+0000 7fefa2400700 1 -- 172.21.15.90:0/2172260737 wait complete. 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.659+0000 7fefa2400700 1 Processor -- start 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.659+0000 7fefa2400700 1 -- start start 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 0x7fef9c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 0x7fef9c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:55.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c115870 con 0x7fef9c075860 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1156f0 con 0x7fef9c076180 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa2400700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1159f0 con 0x7fef9c10fd20 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa1000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50856/0 (socket says 172.21.15.90:50856) 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa1000700 1 -- 172.21.15.90:0/3304705478 learned_addr learned my addr 172.21.15.90:0/3304705478 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fef93e00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 0x7fef9c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.660+0000 7fefa1a00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 0x7fef9c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 -- 172.21.15.90:0/3304705478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 msgr2=0x7fef9c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:55.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 0x7fef9c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 -- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 msgr2=0x7fef9c19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 -- 172.21.15.90:0/3304705478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a0d60 con 0x7fef9c076180 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fefa1000700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fefa1a00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 0x7fef9c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef93e00700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 0x7fef9c19c2e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fef8800e9f0 tx=0x7fef8800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8800c9c0 con 0x7fef9c076180 2024-06-02T15:57:55.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a1050 con 0x7fef9c076180 2024-06-02T15:57:55.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8800cb60 con 0x7fef9c076180 2024-06-02T15:57:55.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88018680 con 0x7fef9c076180 2024-06-02T15:57:55.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.661+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a88d0 con 0x7fef9c076180 2024-06-02T15:57:55.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.663+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c06a230 con 0x7fef9c076180 2024-06-02T15:57:55.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.665+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef88010430 con 0x7fef9c076180 2024-06-02T15:57:55.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.666+0000 7fef9a000700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 0x7fef780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:57:55.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.667+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fef880e2a50 con 0x7fef9c076180 2024-06-02T15:57:55.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.667+0000 7fefa1000700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 0x7fef780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:57:55.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.667+0000 7fefa1000700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 0x7fef780b4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fef80011590 tx=0x7fef80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:57:55.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.669+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef880aa360 con 0x7fef9c076180 2024-06-02T15:57:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.916+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fef9c19d0b0 con 0x7fef9c076180 2024-06-02T15:57:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.917+0000 7fef9a000700 1 -- 172.21.15.90:0/3304705478 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+590 (secure 0 0 0) 0x7fef880a9c60 con 0x7fef9c076180 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2134019795 2024-06-03T15:29:57.531987+0000 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3164241316 2024-06-03T15:29:57.531987+0000 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/232748990 2024-06-03T15:29:23.308098+0000 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/828520669 2024-06-03T15:29:23.308098+0000 2024-06-02T15:57:55.914 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1506181662 2024-06-03T15:29:23.308098+0000 2024-06-02T15:57:55.915 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1560565398 2024-06-03T15:29:57.531987+0000 2024-06-02T15:57:55.915 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:57:55.915 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/4260623570 2024-06-03T15:29:23.308098+0000 2024-06-02T15:57:55.915 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1946492510 2024-06-03T15:29:57.531987+0000 2024-06-02T15:57:55.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.919+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 msgr2=0x7fef780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.919+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 0x7fef780b4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fef80011590 tx=0x7fef80009260 comp rx=0 tx=0).stop 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.919+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 msgr2=0x7fef9c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.919+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 0x7fef9c19c2e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fef8800e9f0 tx=0x7fef8800ed00 comp rx=0 tx=0).stop 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 shutdown_connections 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fef780b2880 0x7fef780b4d40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef9c10fd20 0x7fef9c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef9c075860 0x7fef9c19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 --2- 172.21.15.90:0/3304705478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef9c076180 0x7fef9c19c2e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:57:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 >> 172.21.15.90:0/3304705478 conn(0x7fef9c0fef90 msgr2=0x7fef9c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:57:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 shutdown_connections 2024-06-02T15:57:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:57:55.920+0000 7fefa2400700 1 -- 172.21.15.90:0/3304705478 wait complete. 2024-06-02T15:57:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-06-02T15:57:56.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:55 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3304705478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:57:56.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:55 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3304705478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:57:56.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:55 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3304705478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T15:57:56.785 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-06-02T15:57:56.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:57:56.786 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-06-02T15:57:57.003 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T15:57:57.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:57:57.004 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-02T15:57:57.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:57:57.094 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-06-02T15:57:57.205 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-02T15:57:57.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:57:57.206 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T15:57:57.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:56 smithi090 ceph-mon[31595]: pgmap v923: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:57.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T15:57:57.270 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-06-02T15:57:57.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:56 smithi120 ceph-mon[36451]: pgmap v923: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:57.318 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:56 smithi135 ceph-mon[36839]: pgmap v923: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 9 op/s 2024-06-02T15:57:57.366 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-02T15:57:57.375 INFO:tasks.cephadm:Teardown begin 2024-06-02T15:57:57.376 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-02T15:57:57.376 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:57:57.798 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:57:58.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:57:58 smithi090 ceph-mon[31595]: pgmap v924: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:57:58.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:57:58 smithi120 ceph-mon[36451]: pgmap v924: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:57:58.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:57:58 smithi135 ceph-mon[36839]: pgmap v924: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:58:00.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.253+0000 7f94da800700 1 -- 172.21.15.90:0/3543228371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d4106010 msgr2=0x7f94d4112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:00.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.253+0000 7f94da800700 1 --2- 172.21.15.90:0/3543228371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d4106010 0x7f94d4112520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00b0a0 tx=0x7f94cc00b3b0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 -- 172.21.15.90:0/3543228371 shutdown_connections 2024-06-02T15:58:00.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 --2- 172.21.15.90:0/3543228371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d4106010 0x7f94d4112520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 --2- 172.21.15.90:0/3543228371 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d410b650 0x7f94d410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 --2- 172.21.15.90:0/3543228371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4105650 0x7f94d4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 -- 172.21.15.90:0/3543228371 >> 172.21.15.90:0/3543228371 conn(0x7f94d4101180 msgr2=0x7f94d41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:00.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 -- 172.21.15.90:0/3543228371 shutdown_connections 2024-06-02T15:58:00.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.254+0000 7f94da800700 1 -- 172.21.15.90:0/3543228371 wait complete. 2024-06-02T15:58:00.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 Processor -- start 2024-06-02T15:58:00.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 -- start start 2024-06-02T15:58:00.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:00.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4106010 0x7f94d41a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:00.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d410b650 0x7f94d419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41158e0 con 0x7f94d410b650 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4115760 con 0x7f94d4105650 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.255+0000 7f94da800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4115a60 con 0x7f94d4106010 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:49128/0 (socket says 172.21.15.90:49128) 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 -- 172.21.15.90:0/3368068648 learned_addr learned my addr 172.21.15.90:0/3368068648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9e00700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d410b650 0x7f94d419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:00.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d8a00700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4106010 0x7f94d41a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 -- 172.21.15.90:0/3368068648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4106010 msgr2=0x7f94d41a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4106010 0x7f94d41a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 -- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d410b650 msgr2=0x7f94d419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d410b650 0x7f94d419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.256+0000 7f94d9400700 1 -- 172.21.15.90:0/3368068648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d419c440 con 0x7f94d4105650 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94d9400700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f94c800eb30 tx=0x7f94c800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c800cbd0 con 0x7f94d4105650 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94c80044d0 con 0x7f94d4105650 2024-06-02T15:58:00.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8003e30 con 0x7f94d4105650 2024-06-02T15:58:00.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d419c730 con 0x7f94d4105650 2024-06-02T15:58:00.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.257+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a89c0 con 0x7f94d4105650 2024-06-02T15:58:00.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.258+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d4106d90 con 0x7f94d4105650 2024-06-02T15:58:00.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.260+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f94c8004000 con 0x7f94d4105650 2024-06-02T15:58:00.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.262+0000 7f94d2000700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 0x7f94bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.262+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f94c80e2910 con 0x7f94d4105650 2024-06-02T15:58:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.262+0000 7f94d8a00700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 0x7f94bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.263+0000 7f94d8a00700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 0x7f94bc0b4dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f94d419d260 tx=0x7f94c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:00.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.264+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94c80042f0 con 0x7f94d4105650 2024-06-02T15:58:00.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.547+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94d419cfd0 con 0x7f94d4105650 2024-06-02T15:58:00.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.548+0000 7f94d2000700 1 -- 172.21.15.90:0/3368068648 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11880 (secure 0 0 0) 0x7f94c80aa220 con 0x7f94d4105650 2024-06-02T15:58:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:58:00.545 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:57:09.460895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:58:00.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.551+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 msgr2=0x7f94bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:00.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.551+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 0x7f94bc0b4dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f94d419d260 tx=0x7f94c0009260 comp rx=0 tx=0).stop 2024-06-02T15:58:00.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.551+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 msgr2=0x7f94d41a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:00.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.551+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f94c800eb30 tx=0x7f94c800ee40 comp rx=0 tx=0).stop 2024-06-02T15:58:00.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 shutdown_connections 2024-06-02T15:58:00.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f94bc0b2900 0x7f94bc0b4dc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94d410b650 0x7f94d419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f94d4105650 0x7f94d41a1a40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 --2- 172.21.15.90:0/3368068648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94d4106010 0x7f94d41a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:00.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 >> 172.21.15.90:0/3368068648 conn(0x7f94d4101180 msgr2=0x7f94d41017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:00.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 shutdown_connections 2024-06-02T15:58:00.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:00.552+0000 7f94da800700 1 -- 172.21.15.90:0/3368068648 wait complete. 2024-06-02T15:58:00.550 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 46 2024-06-02T15:58:00.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:00 smithi120 ceph-mon[36451]: pgmap v925: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:58:00.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:00 smithi135 ceph-mon[36839]: pgmap v925: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:58:00.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:00 smithi090 ceph-mon[31595]: pgmap v925: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-06-02T15:58:01.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:01 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3368068648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:01.646 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T15:58:01.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:01 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3368068648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:01.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:01 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3368068648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:02.074 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:02.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:02 smithi090 ceph-mon[31595]: pgmap v926: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-02T15:58:02.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:02 smithi120 ceph-mon[36451]: pgmap v926: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-02T15:58:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:02 smithi135 ceph-mon[36839]: pgmap v926: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-06-02T15:58:04.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 -- 172.21.15.90:0/3267645873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 msgr2=0x7f44f0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 --2- 172.21.15.90:0/3267645873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f0105af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009990 tx=0x7f44e0009ca0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 -- 172.21.15.90:0/3267645873 shutdown_connections 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 --2- 172.21.15.90:0/3267645873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f0112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 --2- 172.21.15.90:0/3267645873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 0x7f44f010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 --2- 172.21.15.90:0/3267645873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f0105af0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 -- 172.21.15.90:0/3267645873 >> 172.21.15.90:0/3267645873 conn(0x7f44f0101160 msgr2=0x7f44f0103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 -- 172.21.15.90:0/3267645873 shutdown_connections 2024-06-02T15:58:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.471+0000 7f44f7200700 1 -- 172.21.15.90:0/3267645873 wait complete. 2024-06-02T15:58:04.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.472+0000 7f44f7200700 1 Processor -- start 2024-06-02T15:58:04.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.472+0000 7f44f7200700 1 -- start start 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f01a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 0x7f44f019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01158c0 con 0x7f44f0105670 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0115740 con 0x7f44f010b670 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f7200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0115a40 con 0x7f44f0106030 2024-06-02T15:58:04.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f01a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f5400700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f01a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44898/0 (socket says 172.21.15.90:44898) 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f5e00700 1 -- 172.21.15.90:0/2731198866 learned_addr learned my addr 172.21.15.90:0/2731198866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.473+0000 7f44f6800700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 0x7f44f019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 -- 172.21.15.90:0/2731198866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 msgr2=0x7f44f019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 0x7f44f019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 -- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 msgr2=0x7f44f01a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:04.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f01a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 -- 172.21.15.90:0/2731198866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f019c460 con 0x7f44f0106030 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f5400700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f01a1fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44e00098e0 tx=0x7f44e00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0027070 con 0x7f44f0106030 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f019c6f0 con 0x7f44f0106030 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e0022470 con 0x7f44f0106030 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0017650 con 0x7f44f0106030 2024-06-02T15:58:04.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.474+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a89e0 con 0x7f44f0106030 2024-06-02T15:58:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.476+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f019ca50 con 0x7f44f0106030 2024-06-02T15:58:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.478+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f44e002a070 con 0x7f44f0106030 2024-06-02T15:58:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.479+0000 7f44eaa00700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 0x7f44d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:04.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.479+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f44e00e3350 con 0x7f44f0106030 2024-06-02T15:58:04.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.479+0000 7f44f5e00700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 0x7f44d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:04.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.480+0000 7f44f5e00700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 0x7f44d80b4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005950 tx=0x7f44e400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:04.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.484+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44e0013070 con 0x7f44f0106030 2024-06-02T15:58:04.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:04 smithi090 ceph-mon[31595]: pgmap v927: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:04.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.768+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44f006a230 con 0x7f44f0106030 2024-06-02T15:58:04.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.770+0000 7f44eaa00700 1 -- 172.21.15.90:0/2731198866 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11880 (secure 0 0 0) 0x7f44e00aad50 con 0x7f44f0106030 2024-06-02T15:58:04.767 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:58:04.767 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/661907319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":661907319},{"type":"v1","addr":"172.21.15.90:6835","nonce":661907319}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/3349102106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3349102106},{"type":"v1","addr":"172.21.15.135:6833","nonce":3349102106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/3322691613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3322691613},{"type":"v1","addr":"172.21.15.120:6833","nonce":3322691613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/2238315985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2238315985},{"type":"v1","addr":"172.21.15.135:6835","nonce":2238315985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/683051134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":683051134},{"type":"v1","addr":"172.21.15.90:6837","nonce":683051134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":25979,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6837/1118882234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6836","nonce":1118882234},{"type":"v1","addr":"172.21.15.120:6837","nonce":1118882234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T15:38:19.378346+0000","modified":"2024-06-02T15:57:09.460895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24524,"mds_2":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6837/284781637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":284781637},{"type":"v1","addr":"172.21.15.135:6837","nonce":284781637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.120:6835/3465448684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":3465448684},{"type":"v1","addr":"172.21.15.120:6835","nonce":3465448684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6839/3736093015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":3736093015},{"type":"v1","addr":"172.21.15.135:6839","nonce":3736093015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"g","rank":2,"incarnation":22,"state":"up:active","state_seq":27,"addr":"172.21.15.90:6839/1414255295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":1414255295},{"type":"v1","addr":"172.21.15.90:6839","nonce":1414255295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.90:6841/1722073812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":1722073812},{"type":"v1","addr":"172.21.15.90:6841","nonce":1722073812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":249,"addr":"172.21.15.120:6839/3642613472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6838","nonce":3642613472},{"type":"v1","addr":"172.21.15.120:6839","nonce":3642613472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24536,"qdb_cluster":[24536,24512,24524]},"id":1}]} 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 msgr2=0x7f44d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 0x7f44d80b4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005950 tx=0x7f44e400b500 comp rx=0 tx=0).stop 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 msgr2=0x7f44f01a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f01a1fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f44e00098e0 tx=0x7f44e00057c0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 shutdown_connections 2024-06-02T15:58:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f44d80b2880 0x7f44d80b4d40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f44f010b670 0x7f44f019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44f0105670 0x7f44f01a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 --2- 172.21.15.90:0/2731198866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f44f0106030 0x7f44f01a1fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 >> 172.21.15.90:0/2731198866 conn(0x7f44f0101160 msgr2=0x7f44f0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 shutdown_connections 2024-06-02T15:58:04.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:04.773+0000 7f44f7200700 1 -- 172.21.15.90:0/2731198866 wait complete. 2024-06-02T15:58:04.772 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 46 2024-06-02T15:58:04.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:04 smithi120 ceph-mon[36451]: pgmap v927: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:04.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:04 smithi135 ceph-mon[36839]: pgmap v927: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:05.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:05 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/2731198866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:05.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T15:58:05.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:05 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/2731198866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:05.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:05 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/2731198866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T15:58:06.162 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:06 smithi090 ceph-mon[31595]: pgmap v928: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:06.806 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:06 smithi120 ceph-mon[36451]: pgmap v928: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:06.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:06 smithi135 ceph-mon[36839]: pgmap v928: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-06-02T15:58:08.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:08 smithi090 ceph-mon[31595]: pgmap v929: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:58:08.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.504+0000 7f8864200700 1 -- 172.21.15.90:0/1582833451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c10ffe0 msgr2=0x7f885c1123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:08.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.504+0000 7f8864200700 1 --2- 172.21.15.90:0/1582833451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c10ffe0 0x7f885c1123d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8854009f90 tx=0x7f885400b330 comp rx=0 tx=0).stop 2024-06-02T15:58:08.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 -- 172.21.15.90:0/1582833451 shutdown_connections 2024-06-02T15:58:08.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 --2- 172.21.15.90:0/1582833451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c10ffe0 0x7f885c1123d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 --2- 172.21.15.90:0/1582833451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c06bae0 0x7f885c06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 --2- 172.21.15.90:0/1582833451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06c400 0x7f885c10faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 -- 172.21.15.90:0/1582833451 >> 172.21.15.90:0/1582833451 conn(0x7f885c0793b0 msgr2=0x7f885c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:08.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 -- 172.21.15.90:0/1582833451 shutdown_connections 2024-06-02T15:58:08.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.505+0000 7f8864200700 1 -- 172.21.15.90:0/1582833451 wait complete. 2024-06-02T15:58:08.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 Processor -- start 2024-06-02T15:58:08.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 -- start start 2024-06-02T15:58:08.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:08.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c06c400 0x7f885c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:08.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 0x7f885c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f885c115950 con 0x7f885c10ffe0 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f885c1157d0 con 0x7f885c06c400 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.506+0000 7f8864200700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f885c115ad0 con 0x7f885c06bae0 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8861800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 0x7f885c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8861800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:50354/0 (socket says 172.21.15.90:50354) 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8861800700 1 -- 172.21.15.90:0/885349086 learned_addr learned my addr 172.21.15.90:0/885349086 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8860e00700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c06c400 0x7f885c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:08.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 -- 172.21.15.90:0/885349086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 msgr2=0x7f885c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 -- 172.21.15.90:0/885349086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c06c400 msgr2=0x7f885c19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c06c400 0x7f885c19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8862200700 1 -- 172.21.15.90:0/885349086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1a0df0 con 0x7f885c10ffe0 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.507+0000 7f8861800700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f8862200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 0x7f885c1a06b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f885400b6c0 tx=0x7f885400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854021070 con 0x7f885c10ffe0 2024-06-02T15:58:08.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8854007d60 con 0x7f885c10ffe0 2024-06-02T15:58:08.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854005040 con 0x7f885c10ffe0 2024-06-02T15:58:08.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a1080 con 0x7f885c10ffe0 2024-06-02T15:58:08.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.508+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a8960 con 0x7f885c10ffe0 2024-06-02T15:58:08.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.509+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c06a230 con 0x7f885c10ffe0 2024-06-02T15:58:08.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.511+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8854024070 con 0x7f885c10ffe0 2024-06-02T15:58:08.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.513+0000 7f885a000700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 0x7f88440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:08.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.513+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f885400f020 con 0x7f885c10ffe0 2024-06-02T15:58:08.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.513+0000 7f8861800700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 0x7f88440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:08.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.514+0000 7f8861800700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 0x7f88440b4d40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8850009750 tx=0x7f8850009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:08.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.516+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f885402b020 con 0x7f885c10ffe0 2024-06-02T15:58:08.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.753+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f885c02e560 con 0x7f885c10ffe0 2024-06-02T15:58:08.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.758+0000 7f885a000700 1 -- 172.21.15.90:0/885349086 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24751 (secure 0 0 0) 0x7f8854007420 con 0x7f885c10ffe0 2024-06-02T15:58:08.756 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-06-02T15:58:08.757 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":94,"fsid":"935be7dc-20f4-11ef-bc9b-c7b262605968","created":"2024-06-02T15:28:38.117986+0000","modified":"2024-06-02T15:54:20.318013+0000","last_up_change":"2024-06-02T15:36:58.562675+0000","last_in_change":"2024-06-02T15:36:38.985472+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T15:32:41.694560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T15:38:10.691399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T15:38:15.361392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-02T15:38:27.767622+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2ad1d96f-8599-4ee1-9fd1-789f0be13368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6803","nonce":445134562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6805","nonce":445134562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6809","nonce":445134562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":445134562},{"type":"v1","addr":"172.21.15.90:6807","nonce":445134562}]},"public_addr":"172.21.15.90:6803/445134562","cluster_addr":"172.21.15.90:6805/445134562","heartbeat_back_addr":"172.21.15.90:6809/445134562","heartbeat_front_addr":"172.21.15.90:6807/445134562","state":["exists","up"]},{"osd":1,"uuid":"26a9dcac-5eab-4fac-90fa-534adccbf8c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6801","nonce":3632063452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6803","nonce":3632063452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6807","nonce":3632063452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3632063452},{"type":"v1","addr":"172.21.15.120:6805","nonce":3632063452}]},"public_addr":"172.21.15.120:6801/3632063452","cluster_addr":"172.21.15.120:6803/3632063452","heartbeat_back_addr":"172.21.15.120:6807/3632063452","heartbeat_front_addr":"172.21.15.120:6805/3632063452","state":["exists","up"]},{"osd":2,"uuid":"8a650ebd-c0d3-4787-bf26-59e8c5ed88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6801","nonce":3984900996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6803","nonce":3984900996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6807","nonce":3984900996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":3984900996},{"type":"v1","addr":"172.21.15.135:6805","nonce":3984900996}]},"public_addr":"172.21.15.135:6801/3984900996","cluster_addr":"172.21.15.135:6803/3984900996","heartbeat_back_addr":"172.21.15.135:6807/3984900996","heartbeat_front_addr":"172.21.15.135:6805/3984900996","state":["exists","up"]},{"osd":3,"uuid":"276c9e04-fa0c-4005-aef6-72afffc49bab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6811","nonce":3752831765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6813","nonce":3752831765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6817","nonce":3752831765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3752831765},{"type":"v1","addr":"172.21.15.90:6815","nonce":3752831765}]},"public_addr":"172.21.15.90:6811/3752831765","cluster_addr":"172.21.15.90:6813/3752831765","heartbeat_back_addr":"172.21.15.90:6817/3752831765","heartbeat_front_addr":"172.21.15.90:6815/3752831765","state":["exists","up"]},{"osd":4,"uuid":"62594a89-8539-441e-9ae7-fa5938668b30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6809","nonce":3009183394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6811","nonce":3009183394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6815","nonce":3009183394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3009183394},{"type":"v1","addr":"172.21.15.120:6813","nonce":3009183394}]},"public_addr":"172.21.15.120:6809/3009183394","cluster_addr":"172.21.15.120:6811/3009183394","heartbeat_back_addr":"172.21.15.120:6815/3009183394","heartbeat_front_addr":"172.21.15.120:6813/3009183394","state":["exists","up"]},{"osd":5,"uuid":"fd04176c-ed63-4796-99b1-720481048d12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6809","nonce":3338529277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6811","nonce":3338529277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6815","nonce":3338529277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3338529277},{"type":"v1","addr":"172.21.15.135:6813","nonce":3338529277}]},"public_addr":"172.21.15.135:6809/3338529277","cluster_addr":"172.21.15.135:6811/3338529277","heartbeat_back_addr":"172.21.15.135:6815/3338529277","heartbeat_front_addr":"172.21.15.135:6813/3338529277","state":["exists","up"]},{"osd":6,"uuid":"f4ffd454-114d-46e2-9735-49849a2fa18b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6819","nonce":3483502077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6821","nonce":3483502077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6825","nonce":3483502077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3483502077},{"type":"v1","addr":"172.21.15.90:6823","nonce":3483502077}]},"public_addr":"172.21.15.90:6819/3483502077","cluster_addr":"172.21.15.90:6821/3483502077","heartbeat_back_addr":"172.21.15.90:6825/3483502077","heartbeat_front_addr":"172.21.15.90:6823/3483502077","state":["exists","up"]},{"osd":7,"uuid":"a44884e0-5c29-4809-8204-7e00dcbf3285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6817","nonce":1944245064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6819","nonce":1944245064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6823","nonce":1944245064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":1944245064},{"type":"v1","addr":"172.21.15.120:6821","nonce":1944245064}]},"public_addr":"172.21.15.120:6817/1944245064","cluster_addr":"172.21.15.120:6819/1944245064","heartbeat_back_addr":"172.21.15.120:6823/1944245064","heartbeat_front_addr":"172.21.15.120:6821/1944245064","state":["exists","up"]},{"osd":8,"uuid":"03d2ed37-a237-4c17-a69e-ce75f99dc563","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6817","nonce":1073792121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6819","nonce":1073792121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6823","nonce":1073792121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":1073792121},{"type":"v1","addr":"172.21.15.135:6821","nonce":1073792121}]},"public_addr":"172.21.15.135:6817/1073792121","cluster_addr":"172.21.15.135:6819/1073792121","heartbeat_back_addr":"172.21.15.135:6823/1073792121","heartbeat_front_addr":"172.21.15.135:6821/1073792121","state":["exists","up"]},{"osd":9,"uuid":"a0d14146-6d2a-4b23-bbd2-318bf2dad564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6827","nonce":3920640313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6829","nonce":3920640313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6833","nonce":3920640313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":3920640313},{"type":"v1","addr":"172.21.15.90:6831","nonce":3920640313}]},"public_addr":"172.21.15.90:6827/3920640313","cluster_addr":"172.21.15.90:6829/3920640313","heartbeat_back_addr":"172.21.15.90:6833/3920640313","heartbeat_front_addr":"172.21.15.90:6831/3920640313","state":["exists","up"]},{"osd":10,"uuid":"a96be6b1-b9f1-4e56-8394-092362a013ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6825","nonce":1810652587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6827","nonce":1810652587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6831","nonce":1810652587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":1810652587},{"type":"v1","addr":"172.21.15.120:6829","nonce":1810652587}]},"public_addr":"172.21.15.120:6825/1810652587","cluster_addr":"172.21.15.120:6827/1810652587","heartbeat_back_addr":"172.21.15.120:6831/1810652587","heartbeat_front_addr":"172.21.15.120:6829/1810652587","state":["exists","up"]},{"osd":11,"uuid":"7fc0220b-d39c-4245-8784-8334b9d06f67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6825","nonce":359143967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6827","nonce":359143967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6831","nonce":359143967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":359143967},{"type":"v1","addr":"172.21.15.135:6829","nonce":359143967}]},"public_addr":"172.21.15.135:6825/359143967","cluster_addr":"172.21.15.135:6827/359143967","heartbeat_back_addr":"172.21.15.135:6831/359143967","heartbeat_front_addr":"172.21.15.135:6829/359143967","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:31:49.491089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:14.097538+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:32:37.844637+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:10.602005+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:33:36.765350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:01.884245+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:34:36.522838+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:03.589859+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:35:28.968916+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:02.857958+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:30.880109+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T15:36:57.112993+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2134019795":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/3164241316":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6801/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:0/232748990":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/828520669":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1506181662":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6800/1560565398":"2024-06-03T15:29:57.531987+0000","172.21.15.90:6800/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:6801/4260623570":"2024-06-03T15:29:23.308098+0000","172.21.15.90:0/1946492510":"2024-06-03T15:29:57.531987+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T15:58:08.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 msgr2=0x7f88440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:08.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 0x7f88440b4d40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8850009750 tx=0x7f8850009310 comp rx=0 tx=0).stop 2024-06-02T15:58:08.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 msgr2=0x7f885c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:08.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 0x7f885c1a06b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f885400b6c0 tx=0x7f885400cc30 comp rx=0 tx=0).stop 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 shutdown_connections 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f88440b2880 0x7f88440b4d40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f885c10ffe0 0x7f885c1a06b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f885c06bae0 0x7f885c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 --2- 172.21.15.90:0/885349086 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c06c400 0x7f885c19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:08.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.761+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 >> 172.21.15.90:0/885349086 conn(0x7f885c0793b0 msgr2=0x7f885c111450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:08.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.762+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 shutdown_connections 2024-06-02T15:58:08.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:08.762+0000 7f8864200700 1 -- 172.21.15.90:0/885349086 wait complete. 2024-06-02T15:58:08.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:08 smithi120 ceph-mon[36451]: pgmap v929: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:58:08.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:08 smithi135 ceph-mon[36839]: pgmap v929: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-06-02T15:58:09.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:09 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/885349086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:58:09.640 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-02T15:58:09.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:09 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/885349086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:58:09.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:09 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/885349086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T15:58:10.097 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:10.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:10 smithi090 ceph-mon[31595]: pgmap v930: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-02T15:58:10.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:10 smithi120 ceph-mon[36451]: pgmap v930: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-02T15:58:10.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:10 smithi135 ceph-mon[36839]: pgmap v930: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-06-02T15:58:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 -- 172.21.15.90:0/719617910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6230075770 msgr2=0x7f6230075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 --2- 172.21.15.90:0/719617910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6230075770 0x7f6230075b50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6220009990 tx=0x7f6220009ca0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 -- 172.21.15.90:0/719617910 shutdown_connections 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 --2- 172.21.15.90:0/719617910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f623010ff40 0x7f6230112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 --2- 172.21.15.90:0/719617910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6230075770 0x7f6230075b50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 --2- 172.21.15.90:0/719617910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230076090 0x7f623010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 -- 172.21.15.90:0/719617910 >> 172.21.15.90:0/719617910 conn(0x7f623007a6a0 msgr2=0x7f623007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 -- 172.21.15.90:0/719617910 shutdown_connections 2024-06-02T15:58:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.261+0000 7f6237a00700 1 -- 172.21.15.90:0/719617910 wait complete. 2024-06-02T15:58:12.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.262+0000 7f6237a00700 1 Processor -- start 2024-06-02T15:58:12.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 -- start start 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6230076090 0x7f623019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f623010ff40 0x7f62301a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6230115960 con 0x7f623010ff40 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62301157e0 con 0x7f6230075770 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6237a00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6230115ae0 con 0x7f6230076090 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6235000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6235000700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.90:45896/0 (socket says 172.21.15.90:45896) 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6235000700 1 -- 172.21.15.90:0/3101562752 learned_addr learned my addr 172.21.15.90:0/3101562752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6235a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f623010ff40 0x7f62301a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.263+0000 7f6227e00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6230076090 0x7f623019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 -- 172.21.15.90:0/3101562752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6230076090 msgr2=0x7f623019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6230076090 0x7f623019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 -- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f623010ff40 msgr2=0x7f62301a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f623010ff40 0x7f62301a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 -- 172.21.15.90:0/3101562752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62301a0e90 con 0x7f6230075770 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.264+0000 7f6235000700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6220000c00 tx=0x7f62200057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.265+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220027070 con 0x7f6230075770 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.265+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6220022470 con 0x7f6230075770 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.265+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62301a1120 con 0x7f6230075770 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.265+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220017650 con 0x7f6230075770 2024-06-02T15:58:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.265+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a8a00 con 0x7f6230075770 2024-06-02T15:58:12.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.266+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623006a230 con 0x7f6230075770 2024-06-02T15:58:12.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.268+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f622002a070 con 0x7f6230075770 2024-06-02T15:58:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.270+0000 7f622e000700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 0x7f62140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.270+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f62200e3300 con 0x7f6230075770 2024-06-02T15:58:12.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.270+0000 7f6227e00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 0x7f62140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:12.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.271+0000 7f6227e00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 0x7f62140b4dc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6230111bd0 tx=0x7f6218009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.272+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6220013070 con 0x7f6230075770 2024-06-02T15:58:12.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.560+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f623002e560 con 0x7f6230075770 2024-06-02T15:58:12.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.574+0000 7f622e000700 1 -- 172.21.15.90:0/3101562752 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f62200aaca0 con 0x7f6230075770 2024-06-02T15:58:12.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.578+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 msgr2=0x7f62140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:12.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.578+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 0x7f62140b4dc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6230111bd0 tx=0x7f6218009260 comp rx=0 tx=0).stop 2024-06-02T15:58:12.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.578+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 msgr2=0x7f623019bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:12.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.578+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6220000c00 tx=0x7f62200057c0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 shutdown_connections 2024-06-02T15:58:12.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f62140b2900 0x7f62140b4dc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f623010ff40 0x7f62301a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6230075770 0x7f623019bed0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 --2- 172.21.15.90:0/3101562752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6230076090 0x7f623019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:12.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.579+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 >> 172.21.15.90:0/3101562752 conn(0x7f623007a6a0 msgr2=0x7f6230101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:12.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.581+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 shutdown_connections 2024-06-02T15:58:12.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:12.581+0000 7f6237a00700 1 -- 172.21.15.90:0/3101562752 wait complete. 2024-06-02T15:58:12.580 INFO:teuthology.orchestra.run.smithi090.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-02T15:58:12.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-mon[36451]: pgmap v931: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:58:12.807 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-mds[66675]: -- 172.21.15.120:0/13104035 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b88651a40 con 0x563b88624c00 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: starting mds.b at 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:12.808 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-mds[66060]: -- 172.21.15.120:0/2775995973 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f4fef7a40 con 0x559f4fecac00 2024-06-02T15:58:12.809 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 15:58:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: starting mds.e at 2024-06-02T15:58:12.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-mon[36839]: pgmap v931: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:58:12.817 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:12.817 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:12.817 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-mds[67133]: -- 172.21.15.135:0/3506235409 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604934cda40 con 0x5604934a1000 2024-06-02T15:58:12.817 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: starting mds.c at 2024-06-02T15:58:12.818 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:12.818 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:12.818 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-mds[67452]: -- 172.21.15.135:0/1683560021 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f1a0eda40 con 0x558f1a0c0c00 2024-06-02T15:58:12.818 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 15:58:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: starting mds.i at 2024-06-02T15:58:12.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-mon[31595]: pgmap v931: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-06-02T15:58:12.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a[31591]: 2024-06-02T15:58:12.562+0000 7f988fa00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T15:58:13.124 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g[73120]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:13.124 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g[73120]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:13.125 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-mds[73127]: -- 172.21.15.90:0/1918098367 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1a5449a40 con 0x55e1a541c800 2024-06-02T15:58:13.125 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g[73120]: starting mds.g at 2024-06-02T15:58:13.125 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: ignoring --setuser ceph since I am not root 2024-06-02T15:58:13.125 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: ignoring --setgroup ceph since I am not root 2024-06-02T15:58:13.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-mds[73540]: -- 172.21.15.90:0/1358494530 <== mon.2 v2:172.21.15.120:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589c6475a40 con 0x5589c6448800 2024-06-02T15:58:13.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 15:58:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: starting mds.a at 2024-06-02T15:58:13.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-02T15:58:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3101562752' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T15:58:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T15:58:13.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-02T15:58:13.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T15:58:13.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: osdmap e95: 12 total, 12 up, 12 in 2024-06-02T15:58:13.808 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:13 smithi120 ceph-mon[36451]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3101562752' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-02T15:58:13.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T15:58:13.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: osdmap e95: 12 total, 12 up, 12 in 2024-06-02T15:58:13.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:13 smithi135 ceph-mon[36839]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-02T15:58:13.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3101562752' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: osdmap e95: 12 total, 12 up, 12 in 2024-06-02T15:58:13.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:13 smithi090 ceph-mon[31595]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-06-02T15:58:14.001 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:14.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6836/1523033281,v1:172.21.15.135:6838/1523033281] up:boot 2024-06-02T15:58:14.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.135:6837/3893698498,v1:172.21.15.135:6839/3893698498] up:boot 2024-06-02T15:58:14.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6834/871692121,v1:172.21.15.120:6835/871692121] up:boot 2024-06-02T15:58:14.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.120:6838/2312881448,v1:172.21.15.120:6839/2312881448] up:boot 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6839/3861936224,v1:172.21.15.90:6841/3861936224] up:boot 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: mds.? [v2:172.21.15.90:6838/1089221539,v1:172.21.15.90:6840/1089221539] up:boot 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:58:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:58:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:58:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:14 smithi090 ceph-mon[31595]: pgmap v933: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6836/1523033281,v1:172.21.15.135:6838/1523033281] up:boot 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.135:6837/3893698498,v1:172.21.15.135:6839/3893698498] up:boot 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6834/871692121,v1:172.21.15.120:6835/871692121] up:boot 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.120:6838/2312881448,v1:172.21.15.120:6839/2312881448] up:boot 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6839/3861936224,v1:172.21.15.90:6841/3861936224] up:boot 2024-06-02T15:58:15.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: mds.? [v2:172.21.15.90:6838/1089221539,v1:172.21.15.90:6840/1089221539] up:boot 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:15.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:14 smithi120 ceph-mon[36451]: pgmap v933: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:58:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6836/1523033281,v1:172.21.15.135:6838/1523033281] up:boot 2024-06-02T15:58:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.135:6837/3893698498,v1:172.21.15.135:6839/3893698498] up:boot 2024-06-02T15:58:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6834/871692121,v1:172.21.15.120:6835/871692121] up:boot 2024-06-02T15:58:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.120:6838/2312881448,v1:172.21.15.120:6839/2312881448] up:boot 2024-06-02T15:58:15.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6839/3861936224,v1:172.21.15.90:6841/3861936224] up:boot 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: mds.? [v2:172.21.15.90:6838/1089221539,v1:172.21.15.90:6840/1089221539] up:boot 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T15:58:15.067 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: from='mgr.14152 172.21.15.90:0/2996499877' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T15:58:15.068 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:14 smithi135 ceph-mon[36839]: pgmap v933: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-06-02T15:58:16.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 -- 172.21.15.90:0/3873726315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410eeb0 msgr2=0x7f18f4116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:16.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 --2- 172.21.15.90:0/3873726315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410eeb0 0x7f18f4116a80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b0d0 tx=0x7f18ec00b3e0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 -- 172.21.15.90:0/3873726315 shutdown_connections 2024-06-02T15:58:16.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 --2- 172.21.15.90:0/3873726315 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410eeb0 0x7f18f4116a80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 --2- 172.21.15.90:0/3873726315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 --2- 172.21.15.90:0/3873726315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410a780 0x7f18f410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 -- 172.21.15.90:0/3873726315 >> 172.21.15.90:0/3873726315 conn(0x7f18f4076d10 msgr2=0x7f18f4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:16.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 -- 172.21.15.90:0/3873726315 shutdown_connections 2024-06-02T15:58:16.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.317+0000 7f18fc600700 1 -- 172.21.15.90:0/3873726315 wait complete. 2024-06-02T15:58:16.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.318+0000 7f18fc600700 1 Processor -- start 2024-06-02T15:58:16.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.318+0000 7f18fc600700 1 -- start start 2024-06-02T15:58:16.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:16.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:16.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410eeb0 0x7f18f41a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4119d90 con 0x7f18f410a780 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4119c10 con 0x7f18f410eeb0 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fc600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4119f10 con 0x7f18f4109dd0 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49622/0 (socket says 172.21.15.90:49622) 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.90:45274/0 (socket says 172.21.15.90:45274) 2024-06-02T15:58:16.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9200700 1 -- 172.21.15.90:0/734147042 learned_addr learned my addr 172.21.15.90:0/734147042 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18fa600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410eeb0 0x7f18f41a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 -- 172.21.15.90:0/734147042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410eeb0 msgr2=0x7f18f41a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410eeb0 0x7f18f41a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 -- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 msgr2=0x7f18f41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9c00700 1 -- 172.21.15.90:0/734147042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f41a5230 con 0x7f18f4109dd0 2024-06-02T15:58:16.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.319+0000 7f18f9200700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18f9c00700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e9f0 tx=0x7f18e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800c9c0 con 0x7f18f4109dd0 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e800cb60 con 0x7f18f4109dd0 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f41a5520 con 0x7f18f4109dd0 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8018680 con 0x7f18f4109dd0 2024-06-02T15:58:16.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.320+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f41acda0 con 0x7f18f4109dd0 2024-06-02T15:58:16.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.321+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f406a230 con 0x7f18f4109dd0 2024-06-02T15:58:16.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.324+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f18e8010430 con 0x7f18f4109dd0 2024-06-02T15:58:16.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.325+0000 7f18f2000700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 0x7f18dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:16.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.325+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f18e80e2bf0 con 0x7f18f4109dd0 2024-06-02T15:58:16.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.326+0000 7f18f9200700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 0x7f18dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:16.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.326+0000 7f18f9200700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 0x7f18dc0b4d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f18f4109330 tx=0x7f18e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:16.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.327+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18e80aa400 con 0x7f18f4109dd0 2024-06-02T15:58:16.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:16.614+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f18f402e560 con 0x7f18f4109dd0 2024-06-02T15:58:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:16 smithi090 ceph-mon[31595]: pgmap v934: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-06-02T15:58:16.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:16 smithi120 ceph-mon[36451]: pgmap v934: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-06-02T15:58:16.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:16 smithi135 ceph-mon[36839]: pgmap v934: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s 2024-06-02T15:58:17.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.395+0000 7f18f2000700 1 -- 172.21.15.90:0/734147042 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f18e80a9d00 con 0x7f18f4109dd0 2024-06-02T15:58:17.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 msgr2=0x7f18dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 0x7f18dc0b4d40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f18f4109330 tx=0x7f18e0009260 comp rx=0 tx=0).stop 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 msgr2=0x7f18f41a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18e800e9f0 tx=0x7f18e800ed00 comp rx=0 tx=0).stop 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 shutdown_connections 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f18dc0b2880 0x7f18dc0b4d40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f18f410eeb0 0x7f18f41a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f18f4109dd0 0x7f18f41a0220 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 --2- 172.21.15.90:0/734147042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f410a780 0x7f18f41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.399+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 >> 172.21.15.90:0/734147042 conn(0x7f18f4076d10 msgr2=0x7f18f4077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.400+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 shutdown_connections 2024-06-02T15:58:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:17.400+0000 7f18fc600700 1 -- 172.21.15.90:0/734147042 wait complete. 2024-06-02T15:58:17.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:17 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/734147042' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:17.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:17 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:17.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:17 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/734147042' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:17.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:17 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:17 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/734147042' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:17.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:17 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T15:58:18.255 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-02T15:58:18.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T15:58:18.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T15:58:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: Cluster is now healthy 2024-06-02T15:58:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:58:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: fsmap 12 up:standby 2024-06-02T15:58:18.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:18 smithi090 ceph-mon[31595]: pgmap v935: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:18.706 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:18.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T15:58:18.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T15:58:18.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: Cluster is now healthy 2024-06-02T15:58:18.810 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:58:18.811 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: fsmap 12 up:standby 2024-06-02T15:58:18.811 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:18 smithi120 ceph-mon[36451]: pgmap v935: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T15:58:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T15:58:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: Cluster is now healthy 2024-06-02T15:58:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T15:58:18.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: fsmap 12 up:standby 2024-06-02T15:58:18.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:18 smithi135 ceph-mon[36839]: pgmap v935: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:20.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:20 smithi090 ceph-mon[31595]: pgmap v936: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:20.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:20 smithi120 ceph-mon[36451]: pgmap v936: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:20.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:20 smithi135 ceph-mon[36839]: pgmap v936: 129 pgs: 129 active+clean; 314 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-06-02T15:58:20.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 -- 172.21.15.90:0/789239579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581041a0 msgr2=0x7fdc581081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:20.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 --2- 172.21.15.90:0/789239579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581041a0 0x7fdc581081c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc440098f0 tx=0x7fdc44009c00 comp rx=0 tx=0).stop 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 -- 172.21.15.90:0/789239579 shutdown_connections 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 --2- 172.21.15.90:0/789239579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581088f0 0x7fdc58110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 --2- 172.21.15.90:0/789239579 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 --2- 172.21.15.90:0/789239579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581041a0 0x7fdc581081c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 -- 172.21.15.90:0/789239579 >> 172.21.15.90:0/789239579 conn(0x7fdc5807a6a0 msgr2=0x7fdc5807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.962+0000 7fdc5d800700 1 -- 172.21.15.90:0/789239579 shutdown_connections 2024-06-02T15:58:20.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.963+0000 7fdc5d800700 1 -- 172.21.15.90:0/789239579 wait complete. 2024-06-02T15:58:20.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 Processor -- start 2024-06-02T15:58:20.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 -- start start 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 0x7fdc58075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581088f0 0x7fdc580762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58113700 con 0x7fdc581041a0 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58113580 con 0x7fdc581037f0 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.964+0000 7fdc5d800700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58113880 con 0x7fdc581088f0 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 0x7fdc58075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 0x7fdc58075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36450/0 (socket says 172.21.15.90:36450) 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56000700 1 -- 172.21.15.90:0/1719904346 learned_addr learned my addr 172.21.15.90:0/1719904346 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc57400700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581088f0 0x7fdc580762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 -- 172.21.15.90:0/1719904346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581088f0 msgr2=0x7fdc580762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581088f0 0x7fdc580762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 -- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 msgr2=0x7fdc58075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 0x7fdc58075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 -- 172.21.15.90:0/1719904346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc581a8180 con 0x7fdc581037f0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.965+0000 7fdc56a00700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58075840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9f0 tx=0x7fdc4800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.966+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800c9c0 con 0x7fdc581037f0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.966+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4800cb60 con 0x7fdc581037f0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.966+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc581a8470 con 0x7fdc581037f0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.966+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48018680 con 0x7fdc581037f0 2024-06-02T15:58:20.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.966+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc581a89b0 con 0x7fdc581037f0 2024-06-02T15:58:20.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.967+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5806a230 con 0x7fdc581037f0 2024-06-02T15:58:20.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.969+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc48010430 con 0x7fdc581037f0 2024-06-02T15:58:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.971+0000 7fdc4f400700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 0x7fdc3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.971+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdc480e2bf0 con 0x7fdc581037f0 2024-06-02T15:58:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.971+0000 7fdc56000700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 0x7fdc3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.972+0000 7fdc56000700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 0x7fdc3c0b4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc581031e0 tx=0x7fdc44019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:20.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:20.973+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc480aa400 con 0x7fdc581037f0 2024-06-02T15:58:21.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.206+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdc5802e7a0 con 0x7fdc581037f0 2024-06-02T15:58:21.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.400+0000 7fdc4f400700 1 -- 172.21.15.90:0/1719904346 <== mon.2 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7fdc480a9d00 con 0x7fdc581037f0 2024-06-02T15:58:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 msgr2=0x7fdc3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 0x7fdc3c0b4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc581031e0 tx=0x7fdc44019040 comp rx=0 tx=0).stop 2024-06-02T15:58:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 msgr2=0x7fdc58075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:21.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58075840 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e9f0 tx=0x7fdc4800ed00 comp rx=0 tx=0).stop 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 shutdown_connections 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7fdc3c0b2880 0x7fdc3c0b4d40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc581088f0 0x7fdc580762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc581037f0 0x7fdc58075840 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 --2- 172.21.15.90:0/1719904346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc581041a0 0x7fdc58075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.407+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 >> 172.21.15.90:0/1719904346 conn(0x7fdc5807a6a0 msgr2=0x7fdc58101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:21.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.408+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 shutdown_connections 2024-06-02T15:58:21.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:21.408+0000 7fdc5d800700 1 -- 172.21.15.90:0/1719904346 wait complete. 2024-06-02T15:58:21.408 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' removed 2024-06-02T15:58:21.481 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:21 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1719904346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:21.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:21 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:21 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1719904346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:21.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:21 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:21 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1719904346' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:21.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:21 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:22.697 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:22 smithi090 ceph-mon[31595]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:22.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:22 smithi090 ceph-mon[31595]: osdmap e96: 12 total, 12 up, 12 in 2024-06-02T15:58:22.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:22 smithi090 ceph-mon[31595]: pgmap v938: 97 pgs: 97 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:22.702 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-02T15:58:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:22 smithi120 ceph-mon[36451]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:22 smithi120 ceph-mon[36451]: osdmap e96: 12 total, 12 up, 12 in 2024-06-02T15:58:22.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:22 smithi120 ceph-mon[36451]: pgmap v938: 97 pgs: 97 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:22 smithi135 ceph-mon[36839]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:22 smithi135 ceph-mon[36839]: osdmap e96: 12 total, 12 up, 12 in 2024-06-02T15:58:22.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:22 smithi135 ceph-mon[36839]: pgmap v938: 97 pgs: 97 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:23.135 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:24.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:24 smithi090 ceph-mon[31595]: pgmap v939: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:24.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:24 smithi120 ceph-mon[36451]: pgmap v939: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:24.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:24 smithi135 ceph-mon[36839]: pgmap v939: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:25.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 -- 172.21.15.90:0/4126710031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 msgr2=0x7f2064075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:25.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 --2- 172.21.15.90:0/4126710031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f2064075b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2058009990 tx=0x7f2058009ca0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 -- 172.21.15.90:0/4126710031 shutdown_connections 2024-06-02T15:58:25.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 --2- 172.21.15.90:0/4126710031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 0x7f2064112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 --2- 172.21.15.90:0/4126710031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f2064075b50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 --2- 172.21.15.90:0/4126710031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 0x7f206410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 -- 172.21.15.90:0/4126710031 >> 172.21.15.90:0/4126710031 conn(0x7f206407a6a0 msgr2=0x7f206407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:25.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.611+0000 7f206c600700 1 -- 172.21.15.90:0/4126710031 shutdown_connections 2024-06-02T15:58:25.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.612+0000 7f206c600700 1 -- 172.21.15.90:0/4126710031 wait complete. 2024-06-02T15:58:25.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.613+0000 7f206c600700 1 Processor -- start 2024-06-02T15:58:25.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.613+0000 7f206c600700 1 -- start start 2024-06-02T15:58:25.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.613+0000 7f206c600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.613+0000 7f206c600700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 0x7f206410c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.613+0000 7f206c600700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 0x7f206410ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f206c600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2064115940 con 0x7f2064075770 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f206c600700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20641157c0 con 0x7f206410ff40 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f206c600700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2064115ac0 con 0x7f2064076090 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36474/0 (socket says 172.21.15.90:36474) 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 -- 172.21.15.90:0/1064574851 learned_addr learned my addr 172.21.15.90:0/1064574851 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f206a600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 0x7f206410ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069200700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 0x7f206410c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 -- 172.21.15.90:0/1064574851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 msgr2=0x7f206410c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 0x7f206410c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 -- 172.21.15.90:0/1064574851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 msgr2=0x7f206410ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 0x7f206410ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.614+0000 7f2069c00700 1 -- 172.21.15.90:0/1064574851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206410d710 con 0x7f2064075770 2024-06-02T15:58:25.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f2069c00700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f20580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:25.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058027070 con 0x7f2064075770 2024-06-02T15:58:25.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2058022470 con 0x7f2064075770 2024-06-02T15:58:25.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058017650 con 0x7f2064075770 2024-06-02T15:58:25.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20641ac740 con 0x7f2064075770 2024-06-02T15:58:25.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.615+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20641acbf0 con 0x7f2064075770 2024-06-02T15:58:25.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.617+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2064076e60 con 0x7f2064075770 2024-06-02T15:58:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.619+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2058009d10 con 0x7f2064075770 2024-06-02T15:58:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.621+0000 7f2062000700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 0x7f204c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:25.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.621+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8883+0+0 (secure 0 0 0) 0x7f20580e3250 con 0x7f2064075770 2024-06-02T15:58:25.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.621+0000 7f2069200700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 0x7f204c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:25.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.622+0000 7f2069200700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 0x7f204c0b4dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f206410e0c0 tx=0x7f2050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:25.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.624+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2058013070 con 0x7f2064075770 2024-06-02T15:58:25.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.860+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f206402e560 con 0x7f2064075770 2024-06-02T15:58:25.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.913+0000 7f2062000700 1 -- 172.21.15.90:0/1064574851 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f20580aad10 con 0x7f2064075770 2024-06-02T15:58:25.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.919+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 msgr2=0x7f204c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:25.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.919+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 0x7f204c0b4dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f206410e0c0 tx=0x7f2050009260 comp rx=0 tx=0).stop 2024-06-02T15:58:25.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.919+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 msgr2=0x7f206410f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:25.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.919+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2058000c00 tx=0x7f20580057c0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.920+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 shutdown_connections 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.921+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7f204c0b2900 0x7f204c0b4dc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.921+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f206410ff40 0x7f206410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.921+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2064075770 0x7f206410f850 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.921+0000 7f206c600700 1 --2- 172.21.15.90:0/1064574851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2064076090 0x7f206410c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:25.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.921+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 >> 172.21.15.90:0/1064574851 conn(0x7f206407a6a0 msgr2=0x7f2064100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:25.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.923+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 shutdown_connections 2024-06-02T15:58:25.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:25.923+0000 7f206c600700 1 -- 172.21.15.90:0/1064574851 wait complete. 2024-06-02T15:58:25.928 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' removed 2024-06-02T15:58:26.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:25 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:26.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:25 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:26.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:25 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:26.898 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504 shell --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-06-02T15:58:27.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:26 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:26 smithi090 ceph-mon[31595]: osdmap e97: 12 total, 12 up, 12 in 2024-06-02T15:58:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:26 smithi090 ceph-mon[31595]: pgmap v941: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:26 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:26 smithi120 ceph-mon[36451]: osdmap e97: 12 total, 12 up, 12 in 2024-06-02T15:58:27.307 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:26 smithi120 ceph-mon[36451]: pgmap v941: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:27.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:26 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/1064574851' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:27.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:26 smithi135 ceph-mon[36839]: osdmap e97: 12 total, 12 up, 12 in 2024-06-02T15:58:27.316 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:26 smithi135 ceph-mon[36839]: pgmap v941: 33 pgs: 33 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:27.320 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T15:58:28.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:28 smithi090 ceph-mon[31595]: pgmap v942: 33 pgs: 33 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:28.807 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:28 smithi120 ceph-mon[36451]: pgmap v942: 33 pgs: 33 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:28.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:28 smithi135 ceph-mon[36839]: pgmap v942: 33 pgs: 33 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:29.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3128704614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc106030 msgr2=0x7ff6bc112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3128704614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc106030 0x7ff6bc112540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009f90 tx=0x7ff6b400b330 comp rx=0 tx=0).stop 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3128704614 shutdown_connections 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3128704614 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc106030 0x7ff6bc112540 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3128704614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc10b670 0x7ff6bc10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3128704614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc105670 0x7ff6bc105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:29.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3128704614 >> 172.21.15.90:0/3128704614 conn(0x7ff6bc101160 msgr2=0x7ff6bc103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:29.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3128704614 shutdown_connections 2024-06-02T15:58:29.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.755+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3128704614 wait complete. 2024-06-02T15:58:29.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 Processor -- start 2024-06-02T15:58:29.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 -- start start 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc106030 0x7ff6bc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc10b670 0x7ff6bc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc115900 con 0x7ff6bc105670 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc115780 con 0x7ff6bc10b670 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c1e00700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc115a80 con 0x7ff6bc106030 2024-06-02T15:58:29.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.757+0000 7ff6c0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36488/0 (socket says 172.21.15.90:36488) 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 -- 172.21.15.90:0/3490347718 learned_addr learned my addr 172.21.15.90:0/3490347718 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6bbe00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc106030 0x7ff6bc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c1400700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc10b670 0x7ff6bc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 -- 172.21.15.90:0/3490347718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc106030 msgr2=0x7ff6bc19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc106030 0x7ff6bc19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 -- 172.21.15.90:0/3490347718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc10b670 msgr2=0x7ff6bc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:29.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc10b670 0x7ff6bc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 -- 172.21.15.90:0/3490347718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1a0d20 con 0x7ff6bc105670 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.758+0000 7ff6c0a00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000e9a0 tx=0x7ff6b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.759+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000c9c0 con 0x7ff6bc105670 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.759+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b000cb60 con 0x7ff6bc105670 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.759+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1a1010 con 0x7ff6bc105670 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.759+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0018680 con 0x7ff6bc105670 2024-06-02T15:58:29.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.759+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc1a8890 con 0x7ff6bc105670 2024-06-02T15:58:29.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.760+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc06a230 con 0x7ff6bc105670 2024-06-02T15:58:29.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.762+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff6b0010430 con 0x7ff6bc105670 2024-06-02T15:58:29.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.764+0000 7ff6b9600700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 0x7ff6a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T15:58:29.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.764+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7ff6b00e2800 con 0x7ff6bc105670 2024-06-02T15:58:29.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.764+0000 7ff6bbe00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 0x7ff6a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T15:58:29.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.765+0000 7ff6bbe00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 0x7ff6a40b4d40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc10ac10 tx=0x7ff6a8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T15:58:29.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.766+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff6b00aa460 con 0x7ff6bc105670 2024-06-02T15:58:29.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:29.997+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6bc02e560 con 0x7ff6bc105670 2024-06-02T15:58:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.052+0000 7ff6b9600700 1 -- 172.21.15.90:0/3490347718 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7ff6b00a9d60 con 0x7ff6bc105670 2024-06-02T15:58:30.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.056+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 msgr2=0x7ff6a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:30.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.056+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 0x7ff6a40b4d40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc10ac10 tx=0x7ff6a8009310 comp rx=0 tx=0).stop 2024-06-02T15:58:30.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 msgr2=0x7ff6bc19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T15:58:30.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000e9a0 tx=0x7ff6b000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 shutdown_connections 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:6800/436264481,v1:172.21.15.90:6801/436264481] conn(0x7ff6a40b2880 0x7ff6a40b4d40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6bc10b670 0x7ff6bc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6bc105670 0x7ff6bc19bd10 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000e9a0 tx=0x7ff6b000ecb0 comp rx=0 tx=0).stop 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 --2- 172.21.15.90:0/3490347718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6bc106030 0x7ff6bc19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T15:58:30.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.057+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 >> 172.21.15.90:0/3490347718 conn(0x7ff6bc101160 msgr2=0x7ff6bc1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T15:58:30.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.061+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 shutdown_connections 2024-06-02T15:58:30.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-06-02T15:58:30.062+0000 7ff6c1e00700 1 -- 172.21.15.90:0/3490347718 wait complete. 2024-06-02T15:58:30.062 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' removed 2024-06-02T15:58:30.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:30 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:30.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:30 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:30.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:30 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T15:58:30.989 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:58:31.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:58:31.087 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T15:58:31.118 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-02T15:58:31.118 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T15:58:31.170 DEBUG:teuthology.orchestra.run.smithi120:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T15:58:31.187 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T15:58:31.202 INFO:tasks.cephadm:Stopping all daemons... 2024-06-02T15:58:31.203 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-02T15:58:31.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a 2024-06-02T15:58:31.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 ceph-mon[31595]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:31.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 ceph-mon[31595]: osdmap e98: 12 total, 12 up, 12 in 2024-06-02T15:58:31.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 ceph-mon[31595]: pgmap v944: 1 pgs: 1 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:31 smithi120 ceph-mon[36451]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:31 smithi120 ceph-mon[36451]: osdmap e98: 12 total, 12 up, 12 in 2024-06-02T15:58:31.557 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:31 smithi120 ceph-mon[36451]: pgmap v944: 1 pgs: 1 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:31 smithi135 ceph-mon[36839]: from='client.? 172.21.15.90:0/3490347718' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T15:58:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:31 smithi135 ceph-mon[36839]: osdmap e98: 12 total, 12 up, 12 in 2024-06-02T15:58:31.566 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:31 smithi135 ceph-mon[36839]: pgmap v944: 1 pgs: 1 active+clean; 577 KiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T15:58:31.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 systemd[1]: Stopping Ceph mon.a for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:32.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a[31591]: 2024-06-02T15:58:31.899+0000 7f9896800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:32.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a[31591]: 2024-06-02T15:58:31.899+0000 7f9896800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-02T15:58:32.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:32 smithi090 podman[212467]: 2024-06-02 15:58:32.073586881 +0000 UTC m=+0.315059027 container died f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:58:32.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:32 smithi090 podman[212467]: 2024-06-02 15:58:32.561307527 +0000 UTC m=+0.802779692 container cleanup f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-02T15:58:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:32 smithi090 bash[212467]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a 2024-06-02T15:58:33.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Jun 02 15:58:32 smithi090 podman[212480]: 2024-06-02 15:58:32.90498836 +0000 UTC m=+0.820938113 container remove f7de8496ae94fb6b8952fa4020a710d3d6e1513b572e46518948a1912a674ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe) 2024-06-02T15:58:33.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-02T15:58:33.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:33.481 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-02T15:58:33.482 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-02T15:58:33.482 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.b 2024-06-02T15:58:33.795 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:33 smithi120 systemd[1]: Stopping Ceph mon.b for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:34.057 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b[36447]: 2024-06-02T15:58:33.798+0000 7fb01e867700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:34.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b[36447]: 2024-06-02T15:58:33.798+0000 7fb01e867700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-02T15:58:34.058 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:33 smithi120 podman[72347]: 2024-06-02 15:58:33.870232106 +0000 UTC m=+0.192460551 container died ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-02T15:58:34.523 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:34 smithi120 podman[72347]: 2024-06-02 15:58:34.275087413 +0000 UTC m=+0.597315857 container cleanup ba7a62482c94704672b68bdd2032dc0a720eee6d65e92cfeaca7634f0df51885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-06-02T15:58:34.523 INFO:journalctl@ceph.mon.b.smithi120.stdout:Jun 02 15:58:34 smithi120 bash[72347]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-b 2024-06-02T15:58:34.703 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-02T15:58:34.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:34.752 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-02T15:58:34.752 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-02T15:58:34.752 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.c 2024-06-02T15:58:35.066 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:34 smithi135 systemd[1]: Stopping Ceph mon.c for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:35.559 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-c[36821]: 2024-06-02T15:58:35.111+0000 7f7152280700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:35.560 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-c[36821]: 2024-06-02T15:58:35.111+0000 7f7152280700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-02T15:58:35.560 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:35 smithi135 podman[72983]: 2024-06-02 15:58:35.21353631 +0000 UTC m=+0.254930511 container died 6a140f208065e00dd87b9b3cc68c83f83bee2986aee692c7b094e52b897799c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-c, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:58:35.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:35 smithi135 podman[72983]: 2024-06-02 15:58:35.662339831 +0000 UTC m=+0.703734031 container cleanup 6a140f208065e00dd87b9b3cc68c83f83bee2986aee692c7b094e52b897799c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-c, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:58:35.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Jun 02 15:58:35 smithi135 bash[72983]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mon-c 2024-06-02T15:58:36.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-02T15:58:36.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:36.191 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-02T15:58:36.192 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-02T15:58:36.192 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x 2024-06-02T15:58:36.622 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:36 smithi090 systemd[1]: Stopping Ceph mgr.x for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:37.123 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:58:36.772+0000 7effbf400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:37.123 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x[31836]: 2024-06-02T15:58:36.772+0000 7effbf400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-02T15:58:37.123 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:36 smithi090 podman[212587]: 2024-06-02 15:58:36.870266645 +0000 UTC m=+0.237447747 container died 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5) 2024-06-02T15:58:38.019 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:37 smithi090 podman[212587]: 2024-06-02 15:58:37.737209987 +0000 UTC m=+1.104391046 container cleanup 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-02T15:58:38.020 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:37 smithi090 bash[212587]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x 2024-06-02T15:58:38.372 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Jun 02 15:58:38 smithi090 podman[212599]: 2024-06-02 15:58:38.022465344 +0000 UTC m=+1.142697508 container remove 42ce219d02149b701976663ab774aecd0956fda50b8bce6d62cfe1ab00c64363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b7328fed0e2e9ab66b09f8134f3cebfc71214504, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-02T15:58:38.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-02T15:58:38.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:38.668 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-02T15:58:38.668 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-02T15:58:38.668 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y 2024-06-02T15:58:39.006 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:38 smithi120 systemd[1]: Stopping Ceph mgr.y for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:39.307 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 podman[72470]: 2024-06-02 15:58:39.009689545 +0000 UTC m=+0.141842786 container died 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe) 2024-06-02T15:58:39.666 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 podman[72470]: 2024-06-02 15:58:39.423381912 +0000 UTC m=+0.555535146 container cleanup 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_BRANCH=HEAD) 2024-06-02T15:58:39.667 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 bash[72470]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y 2024-06-02T15:58:39.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-02T15:58:39.932 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 podman[72484]: 2024-06-02 15:58:39.667233808 +0000 UTC m=+0.653184990 container remove 59a61b2195a0bde8c0d92c3c3172d319af2b0d880d9cbb4fd9def91ebad75aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:58:39.933 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-06-02T15:58:39.933 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-06-02T15:58:39.933 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 systemd[1]: Stopped Ceph mgr.y for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T15:58:39.933 INFO:journalctl@ceph.mgr.y.smithi120.stdout:Jun 02 15:58:39 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.y.service: Consumed 15.053s CPU time. 2024-06-02T15:58:39.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:39.945 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-02T15:58:39.946 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-02T15:58:39.946 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.z 2024-06-02T15:58:40.286 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:58:40 smithi135 systemd[1]: Stopping Ceph mgr.z for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:40.566 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:58:40 smithi135 podman[73106]: 2024-06-02 15:58:40.288807039 +0000 UTC m=+0.140316195 container died 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-b7328fe, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-02T15:58:40.935 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:58:40 smithi135 podman[73106]: 2024-06-02 15:58:40.704510605 +0000 UTC m=+0.556019754 container cleanup 1d303ccb328dafd38bba44dd4f9c76ac7c283f20f2e4daacc2df4e4a0c2ac566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:58:40.936 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Jun 02 15:58:40 smithi135 bash[73106]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mgr-z 2024-06-02T15:58:41.073 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-02T15:58:41.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:41.113 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-02T15:58:41.113 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-02T15:58:41.113 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.0 2024-06-02T15:58:41.622 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:41 smithi090 systemd[1]: Stopping Ceph osd.0 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:42.123 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0[45806]: 2024-06-02T15:58:41.740+0000 7f938bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:42.123 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0[45806]: 2024-06-02T15:58:41.740+0000 7f938bc00700 -1 osd.0 98 *** Got signal Terminated *** 2024-06-02T15:58:42.123 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0[45806]: 2024-06-02T15:58:41.740+0000 7f938bc00700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:58:47.755 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:47 smithi090 podman[212708]: 2024-06-02 15:58:47.387910372 +0000 UTC m=+5.792506476 container died 6b5b14de6f3ae72d549cc71a7b1ce887d05b6dee773c721f5e29c9963c2dd871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-06-02T15:58:48.087 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:47 smithi090 podman[212708]: 2024-06-02 15:58:47.90241032 +0000 UTC m=+6.307006378 container cleanup 6b5b14de6f3ae72d549cc71a7b1ce887d05b6dee773c721f5e29c9963c2dd871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-02T15:58:48.087 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:47 smithi090 bash[212708]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0 2024-06-02T15:58:48.372 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:48 smithi090 podman[212729]: 2024-06-02 15:58:48.088595798 +0000 UTC m=+0.689623783 container remove 6b5b14de6f3ae72d549cc71a7b1ce887d05b6dee773c721f5e29c9963c2dd871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe) 2024-06-02T15:58:49.162 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:48 smithi090 podman[212818]: 2024-06-02 15:58:48.81837704 +0000 UTC m=+0.032701862 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:58:49.162 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:48 smithi090 podman[212818]: 2024-06-02 15:58:48.940153794 +0000 UTC m=+0.154478656 container create 24c5a8fca002ce2c79a9c0c73d968ee80bdcdd0a20fc0b3f394c26476f12ce1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:58:49.623 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:49 smithi090 podman[212818]: 2024-06-02 15:58:49.270623426 +0000 UTC m=+0.484948291 container init 24c5a8fca002ce2c79a9c0c73d968ee80bdcdd0a20fc0b3f394c26476f12ce1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-02T15:58:49.623 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:49 smithi090 podman[212818]: 2024-06-02 15:58:49.283053282 +0000 UTC m=+0.497378147 container start 24c5a8fca002ce2c79a9c0c73d968ee80bdcdd0a20fc0b3f394c26476f12ce1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-02T15:58:49.623 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:49 smithi090 podman[212818]: 2024-06-02 15:58:49.418852889 +0000 UTC m=+0.633177792 container attach 24c5a8fca002ce2c79a9c0c73d968ee80bdcdd0a20fc0b3f394c26476f12ce1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:58:52.372 INFO:journalctl@ceph.osd.0.smithi090.stdout:Jun 02 15:58:52 smithi090 podman[212818]: 2024-06-02 15:58:52.011173544 +0000 UTC m=+3.225498437 container died 24c5a8fca002ce2c79a9c0c73d968ee80bdcdd0a20fc0b3f394c26476f12ce1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:58:53.027 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-02T15:58:53.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:58:53.140 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-02T15:58:53.141 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-02T15:58:53.141 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.1 2024-06-02T15:58:53.458 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:53 smithi120 systemd[1]: Stopping Ceph osd.1 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:58:53.809 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1[41021]: 2024-06-02T15:58:53.460+0000 7fbce470b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:58:53.810 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1[41021]: 2024-06-02T15:58:53.460+0000 7fbce470b700 -1 osd.1 98 *** Got signal Terminated *** 2024-06-02T15:58:53.810 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1[41021]: 2024-06-02T15:58:53.460+0000 7fbce470b700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:58:59.143 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:58 smithi120 podman[72596]: 2024-06-02 15:58:58.858364251 +0000 UTC m=+5.514016151 container died 3415f422c4cba4e7a55037eae8e38a95938448c794039a1f4daef01470347338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=main-b7328fe) 2024-06-02T15:58:59.527 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:59 smithi120 podman[72596]: 2024-06-02 15:58:59.23767076 +0000 UTC m=+5.893322688 container cleanup 3415f422c4cba4e7a55037eae8e38a95938448c794039a1f4daef01470347338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe) 2024-06-02T15:58:59.527 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:59 smithi120 bash[72596]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1 2024-06-02T15:58:59.807 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:59 smithi120 podman[72613]: 2024-06-02 15:58:59.531215661 +0000 UTC m=+0.668937540 container remove 3415f422c4cba4e7a55037eae8e38a95938448c794039a1f4daef01470347338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-b7328fe) 2024-06-02T15:59:00.165 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:59 smithi120 podman[72700]: 2024-06-02 15:58:59.754740553 +0000 UTC m=+0.017032694 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:00.165 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:58:59 smithi120 podman[72700]: 2024-06-02 15:58:59.882509764 +0000 UTC m=+0.144801946 container create e4f2bb477ded7914621e51abca045fac49375dbbcce5c6cfcc41cd37cdb75534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.build-date=20240521) 2024-06-02T15:59:00.467 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:59:00 smithi120 podman[72700]: 2024-06-02 15:59:00.168963267 +0000 UTC m=+0.431255400 container init e4f2bb477ded7914621e51abca045fac49375dbbcce5c6cfcc41cd37cdb75534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:00.467 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:59:00 smithi120 podman[72700]: 2024-06-02 15:59:00.172595189 +0000 UTC m=+0.434887317 container start e4f2bb477ded7914621e51abca045fac49375dbbcce5c6cfcc41cd37cdb75534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-06-02T15:59:00.467 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:59:00 smithi120 podman[72700]: 2024-06-02 15:59:00.260400154 +0000 UTC m=+0.522692282 container attach e4f2bb477ded7914621e51abca045fac49375dbbcce5c6cfcc41cd37cdb75534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-06-02T15:59:00.807 INFO:journalctl@ceph.osd.1.smithi120.stdout:Jun 02 15:59:00 smithi120 podman[72700]: 2024-06-02 15:59:00.471429393 +0000 UTC m=+0.733721533 container died e4f2bb477ded7914621e51abca045fac49375dbbcce5c6cfcc41cd37cdb75534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-02T15:59:01.171 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-02T15:59:01.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:01.209 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-02T15:59:01.209 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-02T15:59:01.209 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.2 2024-06-02T15:59:01.497 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:01 smithi135 systemd[1]: Stopping Ceph osd.2 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:01.816 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2[41940]: 2024-06-02T15:59:01.500+0000 7f0dc83e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:01.816 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2[41940]: 2024-06-02T15:59:01.500+0000 7f0dc83e8700 -1 osd.2 98 *** Got signal Terminated *** 2024-06-02T15:59:01.816 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2[41940]: 2024-06-02T15:59:01.500+0000 7f0dc83e8700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:07.206 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:06 smithi135 podman[73242]: 2024-06-02 15:59:06.939948296 +0000 UTC m=+5.529705541 container died e9738fe3c28c638b98c69df9e35a6cdd82799576b1d251a1f70c81644ec1b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-06-02T15:59:07.565 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:07 smithi135 podman[73242]: 2024-06-02 15:59:07.342370926 +0000 UTC m=+5.932128150 container cleanup e9738fe3c28c638b98c69df9e35a6cdd82799576b1d251a1f70c81644ec1b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-02T15:59:07.565 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:07 smithi135 bash[73242]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2 2024-06-02T15:59:07.816 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:07 smithi135 podman[73260]: 2024-06-02 15:59:07.568856996 +0000 UTC m=+0.627095946 container remove e9738fe3c28c638b98c69df9e35a6cdd82799576b1d251a1f70c81644ec1b6be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:59:08.169 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:07 smithi135 podman[73349]: 2024-06-02 15:59:07.80061494 +0000 UTC m=+0.017830623 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:08.169 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:07 smithi135 podman[73349]: 2024-06-02 15:59:07.919781379 +0000 UTC m=+0.136997055 container create f01cdf94bc47c405a73fe7a9286335e633906b918732ab9dd2b87cf19aa746f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:59:08.473 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:08 smithi135 podman[73349]: 2024-06-02 15:59:08.172845809 +0000 UTC m=+0.390061501 container init f01cdf94bc47c405a73fe7a9286335e633906b918732ab9dd2b87cf19aa746f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-02T15:59:08.474 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:08 smithi135 podman[73349]: 2024-06-02 15:59:08.176302459 +0000 UTC m=+0.393518142 container start f01cdf94bc47c405a73fe7a9286335e633906b918732ab9dd2b87cf19aa746f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_CLEAN=True) 2024-06-02T15:59:08.474 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:08 smithi135 podman[73349]: 2024-06-02 15:59:08.264328336 +0000 UTC m=+0.481544054 container attach f01cdf94bc47c405a73fe7a9286335e633906b918732ab9dd2b87cf19aa746f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:08.764 INFO:journalctl@ceph.osd.2.smithi135.stdout:Jun 02 15:59:08 smithi135 podman[73349]: 2024-06-02 15:59:08.478164838 +0000 UTC m=+0.695380524 container died f01cdf94bc47c405a73fe7a9286335e633906b918732ab9dd2b87cf19aa746f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-02T15:59:09.123 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-02T15:59:09.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:09.164 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-02T15:59:09.164 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-02T15:59:09.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.3 2024-06-02T15:59:09.593 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:09 smithi090 systemd[1]: Stopping Ceph osd.3 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:09.873 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:09 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:09.769+0000 7fc9d9e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:09.874 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:09 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:09.769+0000 7fc9d9e00700 -1 osd.3 98 *** Got signal Terminated *** 2024-06-02T15:59:09.874 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:09 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:09.769+0000 7fc9d9e00700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:10.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:09.632+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:10.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:09.987+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:10.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:10.200+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:11.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:10.646+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:11.299 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:11 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:11.055+0000 7fc9d5600700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.664668+0000 front 2024-06-02T15:58:44.664093+0000 (oldest deadline 2024-06-02T15:59:10.564287+0000) 2024-06-02T15:59:11.306 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:10.976+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:11.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:11.241+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:12.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:11.636+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:12.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:11.978+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:12.373 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:12 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:12.085+0000 7fc9d5600700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.664668+0000 front 2024-06-02T15:58:44.664093+0000 (oldest deadline 2024-06-02T15:59:10.564287+0000) 2024-06-02T15:59:12.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:12.220+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:13.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:12.631+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:13.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:12.966+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:13.372 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:13 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:13.124+0000 7fc9d5600700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.664668+0000 front 2024-06-02T15:58:44.664093+0000 (oldest deadline 2024-06-02T15:59:10.564287+0000) 2024-06-02T15:59:13.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:13 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:13.113+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:13.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:13.174+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:14.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:13.631+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:14.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:13.934+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:14.372 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3[52746]: 2024-06-02T15:59:14.118+0000 7fc9d5600700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.664668+0000 front 2024-06-02T15:58:44.664093+0000 (oldest deadline 2024-06-02T15:59:10.564287+0000) 2024-06-02T15:59:14.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:14 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:14.147+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:14.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:14.185+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:15.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:14.654+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:15.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:14.943+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:15.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:15.185+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:15.622 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:15 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:15.183+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:15.623 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:15 smithi090 podman[212923]: 2024-06-02 15:59:15.335932073 +0000 UTC m=+5.709941047 container died aa2e5ff461689edd9b07db67ef7aa165c0ef67b2468ba99826fd95dd3aa8bcfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T15:59:16.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:15.628+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:16.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:15 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:15.936+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:16.333 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:16 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:16.163+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:16.333 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:15 smithi090 podman[212923]: 2024-06-02 15:59:15.941436545 +0000 UTC m=+6.315445530 container cleanup aa2e5ff461689edd9b07db67ef7aa165c0ef67b2468ba99826fd95dd3aa8bcfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-02T15:59:16.333 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:15 smithi090 bash[212923]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3 2024-06-02T15:59:16.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:16.146+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:16.622 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:16 smithi090 podman[212953]: 2024-06-02 15:59:16.335750066 +0000 UTC m=+0.986279834 container remove aa2e5ff461689edd9b07db67ef7aa165c0ef67b2468ba99826fd95dd3aa8bcfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-02T15:59:17.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:16.669+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:17.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:16 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:16.924+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:17.463 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:17 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:17.198+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:17.464 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:17 smithi090 podman[213043]: 2024-06-02 15:59:17.067709902 +0000 UTC m=+0.033266467 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:17.464 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:17 smithi090 podman[213043]: 2024-06-02 15:59:17.269590129 +0000 UTC m=+0.235146719 container create 8fc39c80c0395f2bccd3678a965bb2e7b3c3588886693c3cad0070fbca20ab71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-06-02T15:59:17.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:17.194+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:17.873 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:17 smithi090 podman[213043]: 2024-06-02 15:59:17.592778766 +0000 UTC m=+0.558335403 container init 8fc39c80c0395f2bccd3678a965bb2e7b3c3588886693c3cad0070fbca20ab71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-b7328fe) 2024-06-02T15:59:17.873 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:17 smithi090 podman[213043]: 2024-06-02 15:59:17.604353905 +0000 UTC m=+0.569910518 container start 8fc39c80c0395f2bccd3678a965bb2e7b3c3588886693c3cad0070fbca20ab71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:59:17.874 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:17 smithi090 podman[213043]: 2024-06-02 15:59:17.708438887 +0000 UTC m=+0.673995490 container attach 8fc39c80c0395f2bccd3678a965bb2e7b3c3588886693c3cad0070fbca20ab71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-02T15:59:18.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:17.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:18.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:17 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:17.891+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:18.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:18 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:18.247+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:18.373 INFO:journalctl@ceph.osd.3.smithi090.stdout:Jun 02 15:59:18 smithi090 podman[213043]: 2024-06-02 15:59:18.057998327 +0000 UTC m=+1.023554977 container died 8fc39c80c0395f2bccd3678a965bb2e7b3c3588886693c3cad0070fbca20ab71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:59:18.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:18.158+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:18.949 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-02T15:59:19.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:19.052 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-02T15:59:19.052 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-02T15:59:19.052 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.4 2024-06-02T15:59:19.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:18.684+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:19.307 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:19 smithi120 systemd[1]: Stopping Ceph osd.4 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:19.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:18 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:18.910+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:19.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:19.156+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:19.577 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:19.313+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:19.577 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:19.367+0000 7fcb89e36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:19.577 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:19.367+0000 7fcb89e36700 -1 osd.4 98 *** Got signal Terminated *** 2024-06-02T15:59:19.577 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:19.367+0000 7fcb89e36700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:19.577 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:19.579+0000 7fcb8642a700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.555547+0000 front 2024-06-02T15:58:54.555535+0000 (oldest deadline 2024-06-02T15:59:19.255391+0000) 2024-06-02T15:59:19.622 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:19 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:19.211+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:20.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:19.728+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:20.271 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:19 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:19.886+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:20.390 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:20.134+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:20.391 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:20.134+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:20.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:20.274+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:20.622 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:20 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:20.229+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:20.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:20 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:20.322+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:20.847 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:20.563+0000 7fcb8642a700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.555547+0000 front 2024-06-02T15:58:54.555535+0000 (oldest deadline 2024-06-02T15:59:19.255391+0000) 2024-06-02T15:59:21.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:20.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:21.292 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:20 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:20.850+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:21.424 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:21.146+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:21.424 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:21.146+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:21.557 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:21.528+0000 7fcb8642a700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.555547+0000 front 2024-06-02T15:58:54.555535+0000 (oldest deadline 2024-06-02T15:59:19.255391+0000) 2024-06-02T15:59:21.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:21.295+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:21.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:21.293+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:21.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:21.244+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:21.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:21 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:21.244+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:21.752 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:21.427+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:22.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:21.755+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:22.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:21.755+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:22.274 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:21.859+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:22.274 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:21 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:21.859+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:22.423 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:22.175+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:22.559 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:22.175+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:22.559 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:22.276+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:22.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:22 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:22.278+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:22.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:22 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:22.214+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:22.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:22 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:22.214+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:22.763 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:22.427+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:22.895 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:22.564+0000 7fcb8642a700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.555547+0000 front 2024-06-02T15:58:54.555535+0000 (oldest deadline 2024-06-02T15:59:19.255391+0000) 2024-06-02T15:59:23.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:22.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:23.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:22.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:23.251 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:22.898+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:23.251 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:22 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:22.898+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:23.454 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:23.155+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:23.455 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:23.155+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:23.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:23.254+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:23.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:23.326+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:23.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:23.190+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:23.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:23 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:23.190+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:23.763 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:23.458+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:23.864 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4[47785]: 2024-06-02T15:59:23.603+0000 7fcb8642a700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.555547+0000 front 2024-06-02T15:58:54.555535+0000 (oldest deadline 2024-06-02T15:59:19.255391+0000) 2024-06-02T15:59:24.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:23.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:24.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:23.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:24.254 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:23.867+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:24.255 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:23 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:23.867+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:24.486 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:24.194+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:24.486 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:24.194+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:24.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:24.257+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:24.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:24 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:24.316+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:24.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:24 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:24.182+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:24.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:24 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:24.182+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:24.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:24.488+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:24.816 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:24.725+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:24.817 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:24 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:24.725+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:25.009 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:24 smithi120 podman[72816]: 2024-06-02 15:59:24.732170213 +0000 UTC m=+5.484081379 container died e0ae719f760f04c76d7d0664df5a2a71f6352e4ed0ef5e66f6cca9d90f0b189b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-02T15:59:25.009 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:24.871+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:25.009 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:24 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:24.871+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:25.289 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:25 smithi120 podman[72816]: 2024-06-02 15:59:25.110705075 +0000 UTC m=+5.862616241 container cleanup e0ae719f760f04c76d7d0664df5a2a71f6352e4ed0ef5e66f6cca9d90f0b189b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-06-02T15:59:25.289 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:25 smithi120 bash[72816]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4 2024-06-02T15:59:25.437 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:25 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:25.189+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:25.437 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:25 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:25.189+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:25.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:25.291+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:25.557 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:25 smithi120 podman[72832]: 2024-06-02 15:59:25.420943789 +0000 UTC m=+0.689632650 container remove e0ae719f760f04c76d7d0664df5a2a71f6352e4ed0ef5e66f6cca9d90f0b189b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:59:25.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:25 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:25.306+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:25.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:25 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:25.136+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:25.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:25 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:25.136+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:25.694 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:25 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:25.440+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:26.057 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:25 smithi120 podman[72922]: 2024-06-02 15:59:25.693245916 +0000 UTC m=+0.017372800 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:26.058 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:25 smithi120 podman[72922]: 2024-06-02 15:59:25.838856634 +0000 UTC m=+0.162983560 container create 6c745ff7a0bc882cca3fff7e190ec30611da3865e1c535c85d839570c8f60a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.build-date=20240521) 2024-06-02T15:59:26.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:25.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:26.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:25 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:25.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:26.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:25 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:25.697+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:26.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:25 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:25.697+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:26.357 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:26.291+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:26.357 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:26 smithi120 podman[72922]: 2024-06-02 15:59:26.066495002 +0000 UTC m=+0.390621885 container init 6c745ff7a0bc882cca3fff7e190ec30611da3865e1c535c85d839570c8f60a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:26.357 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:26 smithi120 podman[72922]: 2024-06-02 15:59:26.069677564 +0000 UTC m=+0.393804455 container start 6c745ff7a0bc882cca3fff7e190ec30611da3865e1c535c85d839570c8f60a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4-deactivate, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-06-02T15:59:26.357 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:26 smithi120 podman[72922]: 2024-06-02 15:59:26.15825233 +0000 UTC m=+0.482379220 container attach 6c745ff7a0bc882cca3fff7e190ec30611da3865e1c535c85d839570c8f60a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0) 2024-06-02T15:59:26.461 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:26 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:26.169+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:26.462 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:26 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:26.169+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:26.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:26 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:26.300+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:26.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:26 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:26.176+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:26.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:26 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:26.176+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:26.674 INFO:journalctl@ceph.osd.4.smithi120.stdout:Jun 02 15:59:26 smithi120 podman[72922]: 2024-06-02 15:59:26.361000492 +0000 UTC m=+0.685127382 container died 6c745ff7a0bc882cca3fff7e190ec30611da3865e1c535c85d839570c8f60a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-06-02T15:59:26.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:26 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:26.465+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:26.816 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:26 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:26.649+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:26.816 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:26 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:26.649+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:27.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-02T15:59:27.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:27.062 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-02T15:59:27.062 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-02T15:59:27.062 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.5 2024-06-02T15:59:27.271 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:26.937+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:27.271 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:26 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:26.937+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:27.368 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 systemd[1]: Stopping Ceph osd.5 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:27.369 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:27.172+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:27.369 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:27.172+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:27.369 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:27.371+0000 7f8485157700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:27.369 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:27.371+0000 7f8485157700 -1 osd.5 98 *** Got signal Terminated *** 2024-06-02T15:59:27.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:27.274+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:27.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:27 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:27.257+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:27.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:27 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:27.161+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:27.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:27 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:27.161+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:27.663 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:27.371+0000 7f8485157700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:27.663 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:27.486+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:28.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:27.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:28.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:27 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:27.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:28.278 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:27.906+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:28.279 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:27.906+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:28.279 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:27 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:27.906+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:28.491 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:28.148+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:28.492 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:28.148+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:28.492 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:28.148+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:03.484095+0000 front 2024-06-02T15:59:03.483980+0000 (oldest deadline 2024-06-02T15:59:27.584189+0000) 2024-06-02T15:59:28.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:28 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:28.280+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:28.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:28 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:28.240+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:28.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:28 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:28.176+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:28.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:28 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:28.176+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:28.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:28 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:28.176+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:28.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:28.495+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:28.817 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:28.688+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:28.817 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:28 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:28.688+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:29.256 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:28 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:28.915+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:29.256 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:28 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:28.915+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:29.256 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:28 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:28.915+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:29.457 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:29.195+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:29.457 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:29.195+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:29.457 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:29.195+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:03.484095+0000 front 2024-06-02T15:59:03.483980+0000 (oldest deadline 2024-06-02T15:59:27.584189+0000) 2024-06-02T15:59:29.541 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:29 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:29.259+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:29.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:29.236+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:29.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:29.236+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:29.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:29.159+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:29.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:29.159+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:29.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:29 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:29.159+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:29.708 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:29.460+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:29.709 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:29.460+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:30.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:29.711+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:30.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:29 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:29.711+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:30.252 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:29 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:29.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:30.253 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:29 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:29.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:30.253 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:29 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:29.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:30.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:30 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:30.255+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:30.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:30 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:30.255+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:30.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:30.423+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:30.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:30.423+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:30.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:30.202+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:30.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:30.202+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:30.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:30.202+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:03.484095+0000 front 2024-06-02T15:59:03.483980+0000 (oldest deadline 2024-06-02T15:59:27.584189+0000) 2024-06-02T15:59:30.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:30 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:30.281+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:30.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:30 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:30.281+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:30.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:30 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:30.200+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:30.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:30 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:30.200+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:30.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:30 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:30.200+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:31.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:30.708+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:31.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:30.708+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:31.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:30 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:30.708+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:31.282 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:30 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:30.881+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:31.282 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:30 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:30.881+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:31.282 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:30 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:30.881+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:31.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:31 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:31.285+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:31.557 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:31 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:31.285+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:31.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:31.443+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:31.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:31.443+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:31.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:31.248+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:31.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:31.248+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:31.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:31.248+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:03.484095+0000 front 2024-06-02T15:59:03.483980+0000 (oldest deadline 2024-06-02T15:59:27.584189+0000) 2024-06-02T15:59:31.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:31.257+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:31.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:31.257+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:31.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:31.181+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:31.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:31.181+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:31.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:31.181+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:32.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:31.757+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:32.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:31.757+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:32.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:31 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:31.757+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:32.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:31 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:31.913+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:32.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:31 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:31.913+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:32.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:31 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:31.913+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:32.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:32.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:32.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:32.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:32.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:32.216+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.381011+0000 front 2024-06-02T15:58:44.380661+0000 (oldest deadline 2024-06-02T15:59:09.680539+0000) 2024-06-02T15:59:32.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:32.216+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.683070+0000 front 2024-06-02T15:58:57.683086+0000 (oldest deadline 2024-06-02T15:59:19.983012+0000) 2024-06-02T15:59:32.567 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5[48443]: 2024-06-02T15:59:32.216+0000 7f848174b700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:03.484095+0000 front 2024-06-02T15:59:03.483980+0000 (oldest deadline 2024-06-02T15:59:27.584189+0000) 2024-06-02T15:59:32.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:32 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:32.225+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:32.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:32 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:32.225+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:32.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:32 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:32.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:32.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:32 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:32.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:32.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:32 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:32.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:32.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:32.331+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:32.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:32.331+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:33.067 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:32 smithi135 podman[73465]: 2024-06-02 15:59:32.925353489 +0000 UTC m=+5.664579554 container died 04c3808f7c0a3c32637c7ba19be6660c77f1cf434112b07b6ea776bf0341b5e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-06-02T15:59:33.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:32.748+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:33.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:32.748+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:33.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:32 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:32.748+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:33.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:32.898+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:33.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:32.898+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:33.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:32 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:32.898+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:33.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:33 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:33.232+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:33.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:33 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:33.232+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:33.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:33 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:33.121+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:33.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:33 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:33.121+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:33.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:33 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:33.121+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:33.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:33 smithi135 podman[73465]: 2024-06-02 15:59:33.289110963 +0000 UTC m=+6.028337027 container cleanup 04c3808f7c0a3c32637c7ba19be6660c77f1cf434112b07b6ea776bf0341b5e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-06-02T15:59:33.566 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:33 smithi135 bash[73465]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5 2024-06-02T15:59:33.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:33 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:33.404+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:33.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:33 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:33.404+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:33.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:33.378+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:33.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:33.378+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:33.914 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:33 smithi135 podman[73480]: 2024-06-02 15:59:33.581990632 +0000 UTC m=+0.655685592 container remove 04c3808f7c0a3c32637c7ba19be6660c77f1cf434112b07b6ea776bf0341b5e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True) 2024-06-02T15:59:33.915 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:33 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:33.771+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:33.915 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:33 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:33.771+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:33.915 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:33 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:33.771+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:34.174 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:33 smithi135 podman[73567]: 2024-06-02 15:59:33.819849971 +0000 UTC m=+0.017093512 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:34.175 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:33 smithi135 podman[73567]: 2024-06-02 15:59:33.966880763 +0000 UTC m=+0.164124363 container create e4faa94de616719eab1eaaea18c3c51db5b2264afaf85fdf4b56fbdfa5c5369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, GIT_CLEAN=True) 2024-06-02T15:59:34.175 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:34 smithi135 podman[73567]: 2024-06-02 15:59:34.178281518 +0000 UTC m=+0.375525049 container init e4faa94de616719eab1eaaea18c3c51db5b2264afaf85fdf4b56fbdfa5c5369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T15:59:34.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:33.900+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:34.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:33.900+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:34.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:33 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:33.900+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:34.469 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:34 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:34.420+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:34.469 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:34 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:34.420+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:34.470 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:34 smithi135 podman[73567]: 2024-06-02 15:59:34.181644751 +0000 UTC m=+0.378888293 container start e4faa94de616719eab1eaaea18c3c51db5b2264afaf85fdf4b56fbdfa5c5369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-06-02T15:59:34.470 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:34 smithi135 podman[73567]: 2024-06-02 15:59:34.286003411 +0000 UTC m=+0.483246945 container attach e4faa94de616719eab1eaaea18c3c51db5b2264afaf85fdf4b56fbdfa5c5369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:59:34.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:34 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:34.264+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:34.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:34 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:34.264+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:34.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:34 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:34.135+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:34.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:34 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:34.135+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:34.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:34 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:34.135+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:34.770 INFO:journalctl@ceph.osd.5.smithi135.stdout:Jun 02 15:59:34 smithi135 podman[73567]: 2024-06-02 15:59:34.473399599 +0000 UTC m=+0.670643143 container died e4faa94de616719eab1eaaea18c3c51db5b2264afaf85fdf4b56fbdfa5c5369e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-06-02T15:59:34.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:34 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:34.410+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:34.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:34 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:34.410+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:35.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:34 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:34.772+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:35.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:34 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:34.772+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:35.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:34 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:34.772+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:35.190 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-02T15:59:35.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:35.230 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-02T15:59:35.230 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-02T15:59:35.231 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.6 2024-06-02T15:59:35.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:34 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:34.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:35.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:34 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:34.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:35.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:34 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:34.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:35.354 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:35.299+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:35.355 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:35.299+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:35.090+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:35.090+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:35.090+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:35.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:35.392+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:35.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:35.392+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:35.622 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 systemd[1]: Stopping Ceph osd.6 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:35.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:35.401+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:35.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:35.401+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:36.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:35.760+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:36.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:35.760+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:36.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:35 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:35.760+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:36.108 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:35.799+0000 7fb0d6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:36.108 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:35.799+0000 7fb0d6a00700 -1 osd.6 98 *** Got signal Terminated *** 2024-06-02T15:59:36.108 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:35.799+0000 7fb0d6a00700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:36.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:35.927+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:36.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:35.927+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:36.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:35.927+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:36.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:35 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:35.927+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:36.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:36.327+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:36.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:36.327+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:36.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:36.327+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.496770+0000 front 2024-06-02T15:59:10.496633+0000 (oldest deadline 2024-06-02T15:59:35.796887+0000) 2024-06-02T15:59:36.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:36.110+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:36.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:36.110+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:36.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:36 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:36.110+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:36.789 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:36.424+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:36.789 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:36.424+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:36.789 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:36.424+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:36.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:36.389+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:36.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:36.389+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:36.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:36.389+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:37.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:36.792+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:37.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:36.792+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:37.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:36 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:36.792+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:37.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:36.910+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:37.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:36.910+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:37.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:36.910+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:37.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:36 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:36.910+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:37.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:37.345+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:37.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:37.345+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:37.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:37.346+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.496770+0000 front 2024-06-02T15:59:10.496633+0000 (oldest deadline 2024-06-02T15:59:35.796887+0000) 2024-06-02T15:59:37.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:37.122+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:37.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:37.122+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:37.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:37 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:37.122+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:37.809 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:37.434+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:37.811 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:37.434+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:37.812 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:37.434+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:37.815 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:37.470+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:37.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:37.470+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:37.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:37.470+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:38.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:37.811+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:38.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:37.811+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:38.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:37 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:37.811+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:38.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:37.882+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:38.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:37.882+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:38.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:37.882+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:38.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:37 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:37.882+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:38.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:38.317+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:38.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:38.317+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:38.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:38.317+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.496770+0000 front 2024-06-02T15:59:10.496633+0000 (oldest deadline 2024-06-02T15:59:35.796887+0000) 2024-06-02T15:59:38.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:38.150+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:38.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:38.150+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:38.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:38.151+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:38.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:38.151+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:38.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:38.463+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:38.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:38.463+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:38.808 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:38.463+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:38.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:38.462+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:38.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:38.462+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:38.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:38.462+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:39.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:38.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:39.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:38.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:39.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:38.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:39.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:38 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:38.883+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:39.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:38.830+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:39.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:38.830+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:39.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:38 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:38.830+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:39.525 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:39.277+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:39.525 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:39.277+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:39.526 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:39.277+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.496770+0000 front 2024-06-02T15:59:10.496633+0000 (oldest deadline 2024-06-02T15:59:35.796887+0000) 2024-06-02T15:59:39.526 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:39.145+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:39.526 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:39.145+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:39.526 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:39.145+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:39.526 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:39 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:39.145+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:39.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:39.510+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:39.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:39.510+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:39.808 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:39.510+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:39.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:39.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:39.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:39.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:39.816 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:39.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:40.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:39.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:40.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:39.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:40.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:39.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:40.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:39 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:39.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:40.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:39.854+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:40.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:39.854+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:40.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:39 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:39.854+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:40.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:40.266+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:54.890901+0000 front 2024-06-02T15:58:54.890972+0000 (oldest deadline 2024-06-02T15:59:19.591090+0000) 2024-06-02T15:59:40.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:40.266+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.494497+0000 front 2024-06-02T15:59:06.495039+0000 (oldest deadline 2024-06-02T15:59:28.794595+0000) 2024-06-02T15:59:40.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6[59685]: 2024-06-02T15:59:40.266+0000 7fb0cd200700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.496770+0000 front 2024-06-02T15:59:10.496633+0000 (oldest deadline 2024-06-02T15:59:35.796887+0000) 2024-06-02T15:59:40.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:40.117+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:40.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:40.117+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:40.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:40.117+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:40.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:40 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:40.117+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:40.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:40.466+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:40.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:40.466+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:40.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:40.466+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:40.815 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:40.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:40.815 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:40.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:40.815 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:40.442+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:41.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:40.818+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:41.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:40.818+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:41.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:40 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:40.818+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:41.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:40.880+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:41.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:40.880+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:41.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:40.880+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:41.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:40 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:40.880+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:41.372 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:41.114+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:41.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:41.114+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:41.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:41.114+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:41.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:41 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:41.114+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:41.739 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:41 smithi090 podman[213149]: 2024-06-02 15:59:41.424713895 +0000 UTC m=+5.763124727 container died e08dfbe038f1a723305e9b0dd087e197841862546899bebf6ba3654026f778fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-06-02T15:59:41.765 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:41.436+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:41.765 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:41.436+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:41.765 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:41.436+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:41.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:41.428+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:41.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:41.428+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:41.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:41.428+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:41.992 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:41 smithi090 podman[213149]: 2024-06-02 15:59:41.970154117 +0000 UTC m=+6.308564952 container cleanup e08dfbe038f1a723305e9b0dd087e197841862546899bebf6ba3654026f778fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:59:41.992 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:41 smithi090 bash[213149]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6 2024-06-02T15:59:42.069 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:41.768+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:42.070 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:41.768+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:42.070 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:41 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:41.768+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:42.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:41.846+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:42.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:41.846+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:42.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:41.846+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:42.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:41 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:41.846+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:42.373 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:42 smithi090 podman[213165]: 2024-06-02 15:59:42.222138843 +0000 UTC m=+0.783841244 container remove e08dfbe038f1a723305e9b0dd087e197841862546899bebf6ba3654026f778fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-06-02T15:59:42.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:42 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:42.104+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:42.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:42 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:42.104+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:42.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:42 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:42.104+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:42.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:42 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:42.104+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:42.741 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:42.428+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:42.741 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:42.428+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:42.741 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:42.428+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:42.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:42.425+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:42.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:42.425+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:42.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:42.425+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:43.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:42.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:43.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:42.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:43.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:42 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:42.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:43.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:42.866+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:43.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:42.866+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:43.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:42.866+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:43.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:42 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:42.866+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:43.359 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:42 smithi090 podman[213253]: 2024-06-02 15:59:42.901136442 +0000 UTC m=+0.031804418 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:43.360 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:43 smithi090 podman[213253]: 2024-06-02 15:59:43.040087323 +0000 UTC m=+0.170755345 container create 9f26783b0621b86d6c3e0e31a43e260443080be46ace94deccc11e3d88791223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-06-02T15:59:43.360 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:43 smithi090 podman[213253]: 2024-06-02 15:59:43.362308092 +0000 UTC m=+0.492976111 container init 9f26783b0621b86d6c3e0e31a43e260443080be46ace94deccc11e3d88791223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-02T15:59:43.360 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:43 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:43.147+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:43.360 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:43 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:43.147+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:43.360 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:43 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:43.147+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:43.361 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:43 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:43.147+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:43.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:43 smithi090 podman[213253]: 2024-06-02 15:59:43.373967252 +0000 UTC m=+0.504635234 container start 9f26783b0621b86d6c3e0e31a43e260443080be46ace94deccc11e3d88791223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=main-b7328fe, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-02T15:59:43.623 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:43 smithi090 podman[213253]: 2024-06-02 15:59:43.478167392 +0000 UTC m=+0.608835427 container attach 9f26783b0621b86d6c3e0e31a43e260443080be46ace94deccc11e3d88791223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:59:43.735 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:43.402+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:43.735 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:43.402+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:43.736 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:43.402+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:43.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:43.431+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:43.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:43.431+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:43.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:43.431+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:44.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:43.738+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:44.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:43.738+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:44.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:43 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:43.738+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:44.123 INFO:journalctl@ceph.osd.6.smithi090.stdout:Jun 02 15:59:43 smithi090 podman[213279]: 2024-06-02 15:59:43.883327797 +0000 UTC m=+0.034035940 container died 9f26783b0621b86d6c3e0e31a43e260443080be46ace94deccc11e3d88791223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:44.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:43.907+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:44.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:43.907+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:44.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:43.907+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:44.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:43 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:43.907+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:44.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:44.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:44.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:44.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:44.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:44.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:44.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:44 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:44.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:44.677 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-02T15:59:44.697 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:44.396+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:44.698 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:44.396+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:44.698 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:44.396+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:44.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:44.791 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-02T15:59:44.791 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-02T15:59:44.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.7 2024-06-02T15:59:44.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:44.455+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:44.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:44.455+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:44.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:44.455+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:45.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:44.700+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:45.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:44.700+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:45.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:44 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:44.700+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:45.079 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:44 smithi120 systemd[1]: Stopping Ceph osd.7 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:45.080 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:44.903+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:45.080 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:44.903+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:45.080 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:44.903+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:45.080 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:44.903+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:45.474 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.082+0000 7f2436041700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:45.474 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.082+0000 7f2436041700 -1 osd.7 98 *** Got signal Terminated *** 2024-06-02T15:59:45.474 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.082+0000 7f2436041700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:45.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:45 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:45.187+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:45.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:45 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:45.187+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:45.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:45 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:45.187+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:45.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:45 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:45.187+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:45.673 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:45.352+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:45.673 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:45.352+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:45.673 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:45.352+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:45.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.476+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:45.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.476+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:45.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:45.476+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:46.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:45.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:46.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:45.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:46.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:45 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:45.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:46.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:45.893+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:46.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:45.893+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:46.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:45.893+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:46.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:45 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:45.893+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:46.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:46.318+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:46.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:46.318+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:46.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:46.318+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:46.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:46 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:46.185+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:46.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:46 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:46.185+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:46.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:46 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:46.185+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:46.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:46 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:46.185+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:46.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:46 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:46.185+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:46.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:46.507+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:46.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:46.507+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:46.808 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:46.507+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:47.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:46.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:47.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:46.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:47.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:46 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:46.676+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:47.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:46.875+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:47.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:46.875+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:47.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:46.875+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:47.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:46.875+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:47.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:46 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:46.875+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:47.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:47 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:47.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:47.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:47 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:47.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:47.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:47 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:47.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:47.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:47 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:47.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:47.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:47 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:47.153+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:47.657 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:47.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:47.657 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:47.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:47.658 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:47.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:47.658 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:47.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:47.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:47.548+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:47.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:47.548+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:47.808 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:47.548+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:48.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:47.660+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:48.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:47.660+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:48.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:47 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:47.660+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:48.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:47.872+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:48.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:47.872+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:48.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:47.872+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:48.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:47.872+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:48.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:47 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:47.872+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:48.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:48.166+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:48.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:48.166+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:48.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:48.166+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:48.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:48.166+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:48.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:48 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:48.166+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:48.686 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:48.343+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:48.686 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:48.343+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:48.686 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:48.343+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:48.687 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:48.343+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:48.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:48.521+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:48.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:48.521+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:48.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:48.521+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:48.808 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:48.521+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.626729+0000 front 2024-06-02T15:59:23.626783+0000 (oldest deadline 2024-06-02T15:59:48.326888+0000) 2024-06-02T15:59:49.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:48.688+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:49.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:48.688+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:49.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:48 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:48.688+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:49.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:48.920+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:49.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:48.920+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:49.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:48.920+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:49.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:48.920+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:49.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:48 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:48.920+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:49.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:49.120+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:49.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:49.120+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:49.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:49.120+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:49.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:49.120+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:49.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:49 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:49.120+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:49.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:49.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:49.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:49.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:49.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:49.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:49.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:49.364+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:49.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:49.502+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.422589+0000 front 2024-06-02T15:58:57.422716+0000 (oldest deadline 2024-06-02T15:59:19.122736+0000) 2024-06-02T15:59:49.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:49.502+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.324154+0000 front 2024-06-02T15:59:04.324249+0000 (oldest deadline 2024-06-02T15:59:30.224202+0000) 2024-06-02T15:59:49.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:49.502+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:10.225072+0000 front 2024-06-02T15:59:10.225217+0000 (oldest deadline 2024-06-02T15:59:36.124713+0000) 2024-06-02T15:59:49.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7[54305]: 2024-06-02T15:59:49.502+0000 7f2432635700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.626729+0000 front 2024-06-02T15:59:23.626783+0000 (oldest deadline 2024-06-02T15:59:48.326888+0000) 2024-06-02T15:59:50.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:49.720+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:50.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:49.720+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:50.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:49.720+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:50.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:49 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:49.720+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:50.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:49.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:50.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:49.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:50.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:49.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:50.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:49.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:50.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:49.901+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:50.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:50.169+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:50.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:50.169+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:50.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:50.169+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:50.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:50.169+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:50.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:50 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:50.169+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:50.681 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:50.320+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:50.681 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:50.320+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:50.681 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:50.320+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:50.681 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:50.320+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:50.830 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:50 smithi120 podman[73041]: 2024-06-02 15:59:50.56506124 +0000 UTC m=+5.575352685 container died 3dff34e252b903a906ec7b52f4f8766a89ef843ffc9a83d720c2d65ecee73979 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-02T15:59:51.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:50.684+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:51.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:50.684+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:51.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:50.684+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:51.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:50 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:50.684+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:51.198 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:50 smithi120 podman[73041]: 2024-06-02 15:59:50.933201273 +0000 UTC m=+5.943492697 container cleanup 3dff34e252b903a906ec7b52f4f8766a89ef843ffc9a83d720c2d65ecee73979 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, io.buildah.version=1.33.5) 2024-06-02T15:59:51.198 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:50 smithi120 bash[73041]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7 2024-06-02T15:59:51.198 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:50 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:50.860+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:51.198 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:50 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:50.860+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:51.199 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:50 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:50.860+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:51.199 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:50 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:50.860+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:51.199 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:50 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:50.860+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:51.546 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73058]: 2024-06-02 15:59:51.201450496 +0000 UTC m=+0.634089283 container remove 3dff34e252b903a906ec7b52f4f8766a89ef843ffc9a83d720c2d65ecee73979 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-02T15:59:51.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:51.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:51.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:51.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:51.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:51.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:51.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:51.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:51.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:51 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:51.137+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:51.687 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:51.345+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:51.688 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:51.345+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:51.688 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:51.345+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:51.688 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:51.345+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:51.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73147]: 2024-06-02 15:59:51.452668697 +0000 UTC m=+0.017613840 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:51.807 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73147]: 2024-06-02 15:59:51.585902067 +0000 UTC m=+0.150847208 container create 35d9ccf41cb73dfd697a8ca94837da9988d8a4bdcf2eba328c8d5cc964bd6553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-06-02T15:59:52.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:51.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:52.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:51.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:52.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:51.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:52.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:51 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:51.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:52.161 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:51 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:51.810+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:52.162 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:51 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:51.810+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:52.162 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:51 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:51.810+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:52.162 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:51 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:51.810+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:52.162 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:51 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:51.810+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:52.162 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73147]: 2024-06-02 15:59:51.858278038 +0000 UTC m=+0.423223180 container init 35d9ccf41cb73dfd697a8ca94837da9988d8a4bdcf2eba328c8d5cc964bd6553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:52.163 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73147]: 2024-06-02 15:59:51.861318554 +0000 UTC m=+0.426263703 container start 35d9ccf41cb73dfd697a8ca94837da9988d8a4bdcf2eba328c8d5cc964bd6553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T15:59:52.163 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:51 smithi120 podman[73147]: 2024-06-02 15:59:51.938621804 +0000 UTC m=+0.503566970 container attach 35d9ccf41cb73dfd697a8ca94837da9988d8a4bdcf2eba328c8d5cc964bd6553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T15:59:52.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:52.094+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:52.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:52.094+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:52.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:52.094+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:52.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:52.094+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:52.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:52 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:52.094+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:52.434 INFO:journalctl@ceph.osd.7.smithi120.stdout:Jun 02 15:59:52 smithi120 podman[73147]: 2024-06-02 15:59:52.165330879 +0000 UTC m=+0.730276028 container died 35d9ccf41cb73dfd697a8ca94837da9988d8a4bdcf2eba328c8d5cc964bd6553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T15:59:52.663 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:52.326+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:52.663 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:52.326+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:52.663 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:52.326+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:52.664 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:52.326+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:52.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-02T15:59:52.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T15:59:52.861 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-02T15:59:52.861 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-02T15:59:52.861 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.8 2024-06-02T15:59:53.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:52 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:52.783+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:53.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:52 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:52.783+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:53.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:52 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:52.783+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:53.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:52 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:52.783+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:53.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:52 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:52.783+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:53.066 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:52 smithi135 systemd[1]: Stopping Ceph osd.8 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T15:59:53.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:52.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:53.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:52.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:53.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:52.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:53.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:52 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:52.666+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:53.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:53.048+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:53.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:53.048+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:53.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:53.048+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:53.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:53.048+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:53.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:53 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:53.048+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:53.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.175+0000 7f46f7226700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.175+0000 7f46f7226700 -1 osd.8 98 *** Got signal Terminated *** 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.175+0000 7f46f7226700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.333+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.333+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.333+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:53.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:53.333+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:54.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:53.757+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:54.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:53.757+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:54.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:53.757+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:54.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:53.757+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:54.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:53.757+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:54.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:53.653+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:54.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:53.653+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:54.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:53.653+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:54.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:53 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:53.653+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:54.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:54.069+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:54.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:54.069+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:54.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:54.069+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:54.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:54.069+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:54.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:54 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:54.069+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:54.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:54.295+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:54.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:54.295+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:54.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:54.295+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:54.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:54.295+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:55.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:54 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:54.733+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:55.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:54 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:54.733+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:55.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:54 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:54.733+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:55.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:54 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:54.733+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:55.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:54 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:54.733+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:55.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:54.670+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:55.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:54.670+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:55.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:54.670+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:55.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:54.670+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:55.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:54 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:54.670+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:55.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:55.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:55.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:55.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:55.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:55.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:55 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:55.078+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T15:59:55.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:55.292+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:55.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:55.292+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:55.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:55.292+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:55.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:55.292+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:56.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:56.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:56.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:56.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:56.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:56.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:55 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:55.773+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T15:59:56.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:55.644+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:56.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:55.644+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:56.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:55.644+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:56.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:55.644+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:56.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:55 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:55.644+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:56.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:56.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:56.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:56.373 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:56.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:56.374 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:56 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:56.100+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T15:59:56.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:56.290+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:56.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:56.290+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:56.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:56.290+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:56.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:56.290+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:57.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:57.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:57.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:57.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:57.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:57.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:56.792+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T15:59:57.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:56.683+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:57.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:56.683+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:57.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:56.683+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:57.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:56.683+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:57.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:56 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:56.683+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:57.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:57.283+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.447223+0000 front 2024-06-02T15:58:56.447403+0000 (oldest deadline 2024-06-02T15:59:21.147200+0000) 2024-06-02T15:59:57.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:57.283+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:06.348665+0000 front 2024-06-02T15:59:06.348646+0000 (oldest deadline 2024-06-02T15:59:29.248710+0000) 2024-06-02T15:59:57.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:57.283+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:14.450270+0000 front 2024-06-02T15:59:14.450409+0000 (oldest deadline 2024-06-02T15:59:35.550232+0000) 2024-06-02T15:59:57.567 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8[55110]: 2024-06-02T15:59:57.283+0000 7f46ef812700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.751997+0000 front 2024-06-02T15:59:23.751980+0000 (oldest deadline 2024-06-02T15:59:46.652018+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:57.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:57 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:57.146+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T15:59:58.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:58.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:58.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:58.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:58.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:58.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:57 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:57.807+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T15:59:58.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:57.679+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:58.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:57.679+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:58.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:57.679+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:58.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:57.679+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:58.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:57 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:57.679+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:58.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:58.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:58.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:58.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:58.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:58.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:58 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:58.157+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T15:59:58.947 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:58 smithi135 podman[73685]: 2024-06-02 15:59:58.658311128 +0000 UTC m=+5.600088818 container died ed281141d6b413b0053f9bf28e0de42d0705b2bcc444e4af3a2bf1fe94733edf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-06-02T15:59:58.947 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:58 smithi135 podman[73685]: 2024-06-02 15:59:58.950801408 +0000 UTC m=+5.892579097 container cleanup ed281141d6b413b0053f9bf28e0de42d0705b2bcc444e4af3a2bf1fe94733edf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-02T15:59:58.947 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:58 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:58.702+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:58.948 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:58 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:58.702+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:58.948 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:58 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:58.702+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:58.948 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:58 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:58.702+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:58.948 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:58 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:58.702+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:59.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T15:59:59.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T15:59:59.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T15:59:59.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T15:59:59.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T15:59:59.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:58 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:58.785+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T15:59:59.215 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:58 smithi135 bash[73685]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8 2024-06-02T15:59:59.566 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73700]: 2024-06-02 15:59:59.219056789 +0000 UTC m=+0.563714647 container remove ed281141d6b413b0053f9bf28e0de42d0705b2bcc444e4af3a2bf1fe94733edf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-02T15:59:59.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T15:59:59.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T15:59:59.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T15:59:59.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T15:59:59.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T15:59:59.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 15:59:59 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T15:59:59.201+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T15:59:59.827 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:59 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:59.711+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T15:59:59.827 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:59 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:59.712+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T15:59:59.827 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:59 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:59.712+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T15:59:59.827 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:59 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:59.712+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T15:59:59.827 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 15:59:59 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T15:59:59.712+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T15:59:59.828 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73793]: 2024-06-02 15:59:59.478946771 +0000 UTC m=+0.017379606 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T15:59:59.828 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73793]: 2024-06-02 15:59:59.578149433 +0000 UTC m=+0.116582255 container create e576cb11ee432c0b1c8324eb54afcbefbf513546130777c12987de86c648f885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-02T15:59:59.828 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73793]: 2024-06-02 15:59:59.83069815 +0000 UTC m=+0.369130964 container init e576cb11ee432c0b1c8324eb54afcbefbf513546130777c12987de86c648f885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe) 2024-06-02T16:00:00.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:00.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:00.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:00.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:00.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:00.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 15:59:59 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T15:59:59.788+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:00.121 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73793]: 2024-06-02 15:59:59.833881781 +0000 UTC m=+0.372314604 container start e576cb11ee432c0b1c8324eb54afcbefbf513546130777c12987de86c648f885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T16:00:00.121 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 15:59:59 smithi135 podman[73793]: 2024-06-02 15:59:59.914218184 +0000 UTC m=+0.452651005 container attach e576cb11ee432c0b1c8324eb54afcbefbf513546130777c12987de86c648f885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-02T16:00:00.388 INFO:journalctl@ceph.osd.8.smithi135.stdout:Jun 02 16:00:00 smithi135 podman[73793]: 2024-06-02 16:00:00.124664495 +0000 UTC m=+0.663097316 container died e576cb11ee432c0b1c8324eb54afcbefbf513546130777c12987de86c648f885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-06-02T16:00:00.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:00.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:00.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:00.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:00.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:00.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:00.232+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:00.741 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-02T16:00:00.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:00.779 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-02T16:00:00.779 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-02T16:00:00.779 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.9 2024-06-02T16:00:01.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:01.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:01.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:01.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:01.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:01.058 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:00 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:00.808+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:01.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:00 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:00.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:01.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:00 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:00.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:01.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:00 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:00.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:01.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:00 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:00.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:01.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:00 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:00.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:01.123 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:00 smithi090 systemd[1]: Stopping Ceph osd.9 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:01.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:01.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:01.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.226+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.344+0000 7f3539a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.344+0000 7f3539a00700 -1 osd.9 98 *** Got signal Terminated *** 2024-06-02T16:00:01.625 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:01 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:01.344+0000 7f3539a00700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T16:00:02.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:01.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:02.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:01.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:02.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:01.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:02.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:01.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:02.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:01.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:02.310 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:02.310 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:02.310 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:02.310 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:02.310 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:02.311 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:01 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:01.826+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:02.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:02.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:02.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:02.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:02.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:02.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:02 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:02.193+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:03.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:02.785+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:03.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:02.785+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:03.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:02.785+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:03.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:02.785+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:03.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:02 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:02.785+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:03.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:03.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:03.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:03.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:03.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:03.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:02 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:02.836+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:03.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:03.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:03.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:03.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:03.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:03.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:03 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:03.202+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:04.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:03 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:03.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:04.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:03 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:03.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:04.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:03 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:03.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:04.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:03 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:03.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:04.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:03 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:03.751+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:04.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:04.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:04.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:04.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:04.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:04.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:04.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:03 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:03.856+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:04.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:04.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:04.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:04.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:04.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:04.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:04 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:04.210+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:05.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:04 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:04.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:05.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:04 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:04.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:05.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:04 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:04.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:05.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:04 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:04.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:05.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:04 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:04.766+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:05.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:05.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:05.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:05.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:05.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:05.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:05.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:04 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:04.904+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:05.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:05.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:05.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:05.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:05.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:05.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:05 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:05.196+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:06.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:05.762+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:06.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:05.762+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:06.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:05.762+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:06.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:05.762+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:06.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:05.762+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:06.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:06.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:06.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:06.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:06.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:06.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:06.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:05 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:05.855+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:06.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:46.334927+0000 front 2024-06-02T15:58:46.334534+0000 (oldest deadline 2024-06-02T15:59:12.234354+0000) 2024-06-02T16:00:06.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:55.136328+0000 front 2024-06-02T15:58:55.136548+0000 (oldest deadline 2024-06-02T15:59:21.036545+0000) 2024-06-02T16:00:06.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.039424+0000 front 2024-06-02T15:59:05.039496+0000 (oldest deadline 2024-06-02T15:59:27.339663+0000) 2024-06-02T16:00:06.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:12.641791+0000 front 2024-06-02T15:59:12.641976+0000 (oldest deadline 2024-06-02T15:59:37.341231+0000) 2024-06-02T16:00:06.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:21.443008+0000 front 2024-06-02T15:59:21.443153+0000 (oldest deadline 2024-06-02T15:59:45.542885+0000) 2024-06-02T16:00:06.624 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9[66419]: 2024-06-02T16:00:06.178+0000 7f3530200700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:30.244821+0000 front 2024-06-02T15:59:30.244680+0000 (oldest deadline 2024-06-02T15:59:54.944440+0000) 2024-06-02T16:00:07.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:06 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:06.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:07.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:06 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:06.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:07.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:06 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:06.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:07.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:06 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:06.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:07.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:06 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:06.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:07.260 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:06 smithi090 podman[213359]: 2024-06-02 16:00:06.902733385 +0000 UTC m=+5.724154742 container died e07b5d391cc93dd5b8252c664049c2c786a50c54198f88f398f3dd8d59545d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, ceph=True) 2024-06-02T16:00:07.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:07.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:07.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:07.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:07.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:07.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:07.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:06 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:06.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:07.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:07 smithi090 podman[213359]: 2024-06-02 16:00:07.489358643 +0000 UTC m=+6.310779943 container cleanup e07b5d391cc93dd5b8252c664049c2c786a50c54198f88f398f3dd8d59545d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T16:00:07.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:07 smithi090 bash[213359]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9 2024-06-02T16:00:08.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:08.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:08.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:08.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:08.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:08.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:07 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:07.685+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:08.123 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:07 smithi090 podman[213377]: 2024-06-02 16:00:07.691237582 +0000 UTC m=+0.776035231 container remove e07b5d391cc93dd5b8252c664049c2c786a50c54198f88f398f3dd8d59545d57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T16:00:08.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:08.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:08.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:08.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:08.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:08.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:08.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:07 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:07.879+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:08.862 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:08 smithi090 podman[213463]: 2024-06-02 16:00:08.438617944 +0000 UTC m=+0.035939354 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T16:00:08.862 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:08 smithi090 podman[213463]: 2024-06-02 16:00:08.610182293 +0000 UTC m=+0.207503691 container create 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T16:00:08.863 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:08 smithi090 podman[213463]: 2024-06-02 16:00:08.864562232 +0000 UTC m=+0.461883709 container init 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-06-02T16:00:09.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:09.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:09.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:09.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:09.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:09.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:08.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:09.123 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:08 smithi090 podman[213463]: 2024-06-02 16:00:08.875263609 +0000 UTC m=+0.472585076 container start 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe) 2024-06-02T16:00:09.123 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:08 smithi090 podman[213463]: 2024-06-02 16:00:08.963099346 +0000 UTC m=+0.560420786 container attach 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-06-02T16:00:09.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:09.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:09.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:09.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:09.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:09.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:09.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:08 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:08.852+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:09.623 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:09 smithi090 podman[213463]: 2024-06-02 16:00:09.323013653 +0000 UTC m=+0.920335133 container died 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-06-02T16:00:10.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:10.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:10.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:10.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:10.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:10.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:09 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:09.715+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:10.191 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-02T16:00:10.252 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:10 smithi090 podman[213488]: 2024-06-02 16:00:10.150611008 +0000 UTC m=+0.810097370 container remove 50b62b4871077a8f71cf0da81ea97fb2976a0690cc3b58367d9888b137a11d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-06-02T16:00:10.253 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:10 smithi090 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.9.service: Deactivated successfully. 2024-06-02T16:00:10.253 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:10 smithi090 systemd[1]: Stopped Ceph osd.9 for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T16:00:10.253 INFO:journalctl@ceph.osd.9.smithi090.stdout:Jun 02 16:00:10 smithi090 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.9.service: Consumed 2min 16.610s CPU time. 2024-06-02T16:00:10.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:10.287 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-02T16:00:10.287 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-02T16:00:10.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.10 2024-06-02T16:00:10.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:10.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:10.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:10.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:10.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:10.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:10.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:09 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:09.824+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:10.807 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 systemd[1]: Stopping Ceph osd.10 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:11.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:11.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:11.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:11.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:11.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:11.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:10 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:10.704+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:11.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.849+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.877+0000 7f5f16d44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:11.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.877+0000 7f5f16d44700 -1 osd.10 98 *** Got signal Terminated *** 2024-06-02T16:00:11.309 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:10 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:10.877+0000 7f5f16d44700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:12.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:12.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:11 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:11.741+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:12.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:12.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:12.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:12.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:12.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:12.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:12.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:11 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:11.887+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:13.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:13.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:12 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:12.784+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:13.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:13.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:13.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:13.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:13.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:13.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:13.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:12 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:12.921+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:14.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:14.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:14.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:14.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:14.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:14.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:14.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:13 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:13.884+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:14.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:14.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:14.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:14.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:14.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:14.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:14.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:13.825+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:15.307 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:44.453639+0000 front 2024-06-02T15:58:44.453563+0000 (oldest deadline 2024-06-02T15:59:09.153317+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:57.755893+0000 front 2024-06-02T15:58:57.755920+0000 (oldest deadline 2024-06-02T15:59:21.256110+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:04.657316+0000 front 2024-06-02T15:59:04.657295+0000 (oldest deadline 2024-06-02T15:59:27.557493+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-06-02T15:59:13.760605+0000 front 2024-06-02T15:59:13.760723+0000 (oldest deadline 2024-06-02T15:59:35.460455+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:22.461694+0000 front 2024-06-02T15:59:22.461646+0000 (oldest deadline 2024-06-02T15:59:45.961727+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:31.262811+0000 front 2024-06-02T15:59:31.262663+0000 (oldest deadline 2024-06-02T15:59:54.762409+0000) 2024-06-02T16:00:15.308 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:14 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10[61008]: 2024-06-02T16:00:14.890+0000 7f5f0f330700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:40.663448+0000 front 2024-06-02T15:59:40.663599+0000 (oldest deadline 2024-06-02T16:00:03.563092+0000) 2024-06-02T16:00:15.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:15.316 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:15.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:15.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:15.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:15.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:15.317 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:14 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:14.839+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:16.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:16.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:15 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:15.790+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:16.668 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:16 smithi120 podman[73263]: 2024-06-02 16:00:16.407824868 +0000 UTC m=+5.927633880 container died 7aea45ecf563951ddc304b9fca2d62b2c04fdf856959f5d9e6e2e6ad5fe4c796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-06-02T16:00:17.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:16 smithi120 podman[73263]: 2024-06-02 16:00:16.769865398 +0000 UTC m=+6.289674413 container cleanup 7aea45ecf563951ddc304b9fca2d62b2c04fdf856959f5d9e6e2e6ad5fe4c796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T16:00:17.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:16 smithi120 bash[73263]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10 2024-06-02T16:00:17.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:17.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:17.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:17.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:17.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:17.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:17.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:16 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:16.743+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:17.383 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73279]: 2024-06-02 16:00:17.062847338 +0000 UTC m=+0.650319740 container remove 7aea45ecf563951ddc304b9fca2d62b2c04fdf856959f5d9e6e2e6ad5fe4c796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-b7328fe, org.label-schema.license=GPLv2) 2024-06-02T16:00:17.713 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.289362091 +0000 UTC m=+0.017481112 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T16:00:17.713 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.397430921 +0000 UTC m=+0.125549931 container create c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-02T16:00:17.713 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.633110243 +0000 UTC m=+0.361229244 container init c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-06-02T16:00:17.714 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.636165938 +0000 UTC m=+0.364284952 container start c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T16:00:17.714 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.716456523 +0000 UTC m=+0.444575539 container attach c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_BRANCH=HEAD) 2024-06-02T16:00:18.057 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:17 smithi120 podman[73365]: 2024-06-02 16:00:17.930936678 +0000 UTC m=+0.659055692 container died c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T16:00:18.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:18.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:18.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:18.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:18.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:18.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:18.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:17 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:17.767+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:18.479 INFO:journalctl@ceph.osd.10.smithi120.stdout:Jun 02 16:00:18 smithi120 podman[73391]: 2024-06-02 16:00:18.46173951 +0000 UTC m=+0.525003819 container remove c92d42d273ef45fe13f0784bfb61170045a69b2efbcc2718cddd151cada3782c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-06-02T16:00:18.485 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-02T16:00:18.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:18.565 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-02T16:00:18.565 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-02T16:00:18.565 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.11 2024-06-02T16:00:18.867 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 systemd[1]: Stopping Ceph osd.11 for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:18.868 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:18.868 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:18.868 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:18.868 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:18.868 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:18.869 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:18.869 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.756+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:19.182 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.870+0000 7f4e28cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:19.182 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.870+0000 7f4e28cf0700 -1 osd.11 98 *** Got signal Terminated *** 2024-06-02T16:00:19.182 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:18 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:18.870+0000 7f4e28cf0700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T16:00:20.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:20.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:19 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:19.724+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:21.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:21.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:21.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:20 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:20.732+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-02T15:59:55.030099+0000 front 2024-06-02T15:59:55.030140+0000 (oldest deadline 2024-06-02T16:00:19.730066+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:22.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:22.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:21 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:21.701+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-02T15:59:55.030099+0000 front 2024-06-02T15:59:55.030140+0000 (oldest deadline 2024-06-02T16:00:19.730066+0000) 2024-06-02T16:00:23.066 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:23.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:22 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:22.690+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-02T15:59:55.030099+0000 front 2024-06-02T15:59:55.030140+0000 (oldest deadline 2024-06-02T16:00:19.730066+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-06-02T15:58:43.422481+0000 front 2024-06-02T15:58:43.422545+0000 (oldest deadline 2024-06-02T15:59:08.722271+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6804 osd.1 since back 2024-06-02T15:58:56.924282+0000 front 2024-06-02T15:58:56.924172+0000 (oldest deadline 2024-06-02T15:59:21.024204+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-06-02T15:59:05.725071+0000 front 2024-06-02T15:59:05.725152+0000 (oldest deadline 2024-06-02T15:59:30.425238+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6812 osd.4 since back 2024-06-02T15:59:23.327371+0000 front 2024-06-02T15:59:23.327354+0000 (oldest deadline 2024-06-02T15:59:49.227296+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-06-02T15:59:29.227682+0000 front 2024-06-02T15:59:29.227753+0000 (oldest deadline 2024-06-02T15:59:54.527729+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-06-02T15:59:38.028765+0000 front 2024-06-02T15:59:38.028978+0000 (oldest deadline 2024-06-02T16:00:06.828943+0000) 2024-06-02T16:00:24.067 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.120:6820 osd.7 since back 2024-06-02T15:59:46.829441+0000 front 2024-06-02T15:59:46.829574+0000 (oldest deadline 2024-06-02T16:00:11.529311+0000) 2024-06-02T16:00:24.068 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:23 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11[61720]: 2024-06-02T16:00:23.703+0000 7f4e252e4700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-06-02T15:59:55.030099+0000 front 2024-06-02T15:59:55.030140+0000 (oldest deadline 2024-06-02T16:00:19.730066+0000) 2024-06-02T16:00:24.476 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:24 smithi135 podman[73922]: 2024-06-02 16:00:24.210718263 +0000 UTC m=+5.448496531 container died 038c906392331d73ef495da3acca394a62a3f46d0de6180e522060be48d6ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.build-date=20240521) 2024-06-02T16:00:24.816 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:24 smithi135 podman[73922]: 2024-06-02 16:00:24.587395367 +0000 UTC m=+5.825173633 container cleanup 038c906392331d73ef495da3acca394a62a3f46d0de6180e522060be48d6ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-02T16:00:24.817 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:24 smithi135 bash[73922]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11 2024-06-02T16:00:25.202 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:24 smithi135 podman[73940]: 2024-06-02 16:00:24.855365971 +0000 UTC m=+0.634229875 container remove 038c906392331d73ef495da3acca394a62a3f46d0de6180e522060be48d6ee68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T16:00:25.505 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.205909198 +0000 UTC m=+0.113108457 container create c9538dfcbc53cffeb9dc1b70a7eb9cf2d2246af5e9e8cd2ec350069c24ba39c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=main-b7328fe, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T16:00:25.505 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.109826888 +0000 UTC m=+0.017026166 image pull 57e49aa75a3f375d3f00c41cb0b86c82a3db74c6589d24b01ff82e31f6c2f00d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a 2024-06-02T16:00:25.505 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.425706008 +0000 UTC m=+0.332905291 container init c9538dfcbc53cffeb9dc1b70a7eb9cf2d2246af5e9e8cd2ec350069c24ba39c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-02T16:00:25.505 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.428959141 +0000 UTC m=+0.336158409 container start c9538dfcbc53cffeb9dc1b70a7eb9cf2d2246af5e9e8cd2ec350069c24ba39c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T16:00:25.505 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.508386723 +0000 UTC m=+0.415585994 container attach c9538dfcbc53cffeb9dc1b70a7eb9cf2d2246af5e9e8cd2ec350069c24ba39c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-06-02T16:00:25.816 INFO:journalctl@ceph.osd.11.smithi135.stdout:Jun 02 16:00:25 smithi135 podman[74030]: 2024-06-02 16:00:25.720545111 +0000 UTC m=+0.627744390 container died c9538dfcbc53cffeb9dc1b70a7eb9cf2d2246af5e9e8cd2ec350069c24ba39c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-06-02T16:00:26.284 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-02T16:00:26.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:26.321 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-02T16:00:26.321 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-02T16:00:26.321 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.a 2024-06-02T16:00:26.873 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:26 smithi090 systemd[1]: Stopping Ceph mds.a for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:27.373 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:26 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: 2024-06-02T16:00:26.911+0000 7f9b21800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:27.373 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:26 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a[73536]: 2024-06-02T16:00:26.911+0000 7f9b21800700 -1 mds.a *** got signal Terminated *** 2024-06-02T16:00:29.283 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:28 smithi090 podman[213579]: 2024-06-02 16:00:28.894888538 +0000 UTC m=+2.135445180 container died 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-b7328fe, GIT_CLEAN=True) 2024-06-02T16:00:29.534 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:29 smithi090 podman[213579]: 2024-06-02 16:00:29.506439549 +0000 UTC m=+2.746996185 container cleanup 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-06-02T16:00:29.534 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:29 smithi090 bash[213579]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a 2024-06-02T16:00:30.122 INFO:journalctl@ceph.mds.a.smithi090.stdout:Jun 02 16:00:29 smithi090 podman[213591]: 2024-06-02 16:00:29.808065753 +0000 UTC m=+0.892626879 container remove 78c8552a10c11e9a976eabff34fb39fe4a50127897856aa2f61ae0bb67dbf1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-06-02T16:00:30.331 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-02T16:00:30.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:30.437 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-02T16:00:30.437 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-02T16:00:30.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.d 2024-06-02T16:00:30.873 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:30 smithi090 systemd[1]: Stopping Ceph mds.d for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:31.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d[72245]: 2024-06-02T16:00:31.184+0000 7f8c5ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:31.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:31 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d[72245]: 2024-06-02T16:00:31.184+0000 7f8c5ca00700 -1 mds.d *** got signal Terminated *** 2024-06-02T16:00:33.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:33 smithi090 podman[213697]: 2024-06-02 16:00:33.371966549 +0000 UTC m=+2.402313896 container died 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-02T16:00:34.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:33 smithi090 podman[213697]: 2024-06-02 16:00:33.946329659 +0000 UTC m=+2.976677018 container cleanup 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe) 2024-06-02T16:00:34.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:33 smithi090 bash[213697]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d 2024-06-02T16:00:34.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:Jun 02 16:00:34 smithi090 podman[213710]: 2024-06-02 16:00:34.164643768 +0000 UTC m=+0.776383211 container remove 2d69f460223aeb220fa383542b9f5fbcb16ccef32da94986caf0e18448c42cd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-b7328fe, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-02T16:00:34.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-02T16:00:34.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:34.818 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-02T16:00:34.818 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-02T16:00:34.819 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.g 2024-06-02T16:00:35.373 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:34 smithi090 systemd[1]: Stopping Ceph mds.g for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:35.873 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g[73120]: 2024-06-02T16:00:35.397+0000 7fccfae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:35.873 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:35 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g[73120]: 2024-06-02T16:00:35.397+0000 7fccfae00700 -1 mds.g *** got signal Terminated *** 2024-06-02T16:00:37.195 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:36 smithi090 podman[213818]: 2024-06-02 16:00:36.90096691 +0000 UTC m=+1.634833630 container died 52b4d163c9216619e3ce4d256bf216c7355fdfc60186597a65c1b6df5c5b9886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-b7328fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-06-02T16:00:37.460 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:37 smithi090 podman[213818]: 2024-06-02 16:00:37.426236819 +0000 UTC m=+2.160103568 container cleanup 52b4d163c9216619e3ce4d256bf216c7355fdfc60186597a65c1b6df5c5b9886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-02T16:00:37.460 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:37 smithi090 bash[213818]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g 2024-06-02T16:00:37.873 INFO:journalctl@ceph.mds.g.smithi090.stdout:Jun 02 16:00:37 smithi090 podman[213831]: 2024-06-02 16:00:37.679285013 +0000 UTC m=+0.767557811 container remove 52b4d163c9216619e3ce4d256bf216c7355fdfc60186597a65c1b6df5c5b9886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=main-b7328fe) 2024-06-02T16:00:38.160 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-02T16:00:38.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:38.265 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-02T16:00:38.266 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-02T16:00:38.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.j 2024-06-02T16:00:38.873 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:38 smithi090 systemd[1]: Stopping Ceph mds.j for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:39.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j[71791]: 2024-06-02T16:00:38.912+0000 7fde2f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:39.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:38 smithi090 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j[71791]: 2024-06-02T16:00:38.912+0000 7fde2f800700 -1 mds.j *** got signal Terminated *** 2024-06-02T16:00:43.301 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:42 smithi090 podman[213941]: 2024-06-02 16:00:42.952591208 +0000 UTC m=+4.237250290 container died ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-b7328fe, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-02T16:00:43.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:43 smithi090 podman[213941]: 2024-06-02 16:00:43.499957601 +0000 UTC m=+4.784616689 container cleanup ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-02T16:00:43.623 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:43 smithi090 bash[213941]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j 2024-06-02T16:00:44.123 INFO:journalctl@ceph.mds.j.smithi090.stdout:Jun 02 16:00:43 smithi090 podman[213966]: 2024-06-02 16:00:43.776079186 +0000 UTC m=+0.799472832 container remove ae4380e2e9d099cd3bd7d3119a4b7fc19ea79681df88f65a9d39e2d0f28b3b3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-02T16:00:44.275 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-02T16:00:44.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:44.377 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-02T16:00:44.377 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-02T16:00:44.377 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.b 2024-06-02T16:00:44.701 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:44 smithi120 systemd[1]: Stopping Ceph mds.b for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:45.058 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T16:00:44.703+0000 7f29618e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:45.058 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:44 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b[66671]: 2024-06-02T16:00:44.703+0000 7f29618e0700 -1 mds.b *** got signal Terminated *** 2024-06-02T16:00:48.893 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:48 smithi120 podman[73470]: 2024-06-02 16:00:48.640294844 +0000 UTC m=+4.072657816 container died 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-02T16:00:49.160 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:48 smithi120 podman[73470]: 2024-06-02 16:00:48.988494243 +0000 UTC m=+4.420857208 container cleanup 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-02T16:00:49.160 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:48 smithi120 bash[73470]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b 2024-06-02T16:00:49.408 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-02T16:00:49.428 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:49 smithi120 podman[73496]: 2024-06-02 16:00:49.163643107 +0000 UTC m=+0.521259805 container remove 507a63e633085ef89071b4c5cf2baf7aeeddf0db4b17ee57d49970e18eb90314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe) 2024-06-02T16:00:49.429 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:49 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.b.service: Deactivated successfully. 2024-06-02T16:00:49.429 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:49 smithi120 systemd[1]: Stopped Ceph mds.b for 935be7dc-20f4-11ef-bc9b-c7b262605968. 2024-06-02T16:00:49.429 INFO:journalctl@ceph.mds.b.smithi120.stdout:Jun 02 16:00:49 smithi120 systemd[1]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.b.service: Consumed 3min 16.916s CPU time. 2024-06-02T16:00:49.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:49.502 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-02T16:00:49.502 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-02T16:00:49.502 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.e 2024-06-02T16:00:49.807 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:49 smithi120 systemd[1]: Stopping Ceph mds.e for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:50.307 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: 2024-06-02T16:00:49.850+0000 7f829a30c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:50.307 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:49 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e[66056]: 2024-06-02T16:00:49.850+0000 7f829a30c700 -1 mds.e *** got signal Terminated *** 2024-06-02T16:00:52.905 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:52 smithi120 podman[73604]: 2024-06-02 16:00:52.635332028 +0000 UTC m=+2.949881785 container died 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-06-02T16:00:53.273 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:53 smithi120 podman[73604]: 2024-06-02 16:00:53.000122448 +0000 UTC m=+3.314672214 container cleanup 52c643c502e67b1bddaefb7ff5f67f1917fd6432ab6fa6da4bc84962db673512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T16:00:53.273 INFO:journalctl@ceph.mds.e.smithi120.stdout:Jun 02 16:00:53 smithi120 bash[73604]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-e 2024-06-02T16:00:53.408 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-02T16:00:53.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:53.440 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-02T16:00:53.440 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-02T16:00:53.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h 2024-06-02T16:00:53.557 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:53 smithi120 systemd[1]: Stopping Ceph mds.h for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:54.057 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[70102]: 2024-06-02T16:00:53.753+0000 7f9356be7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:54.057 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:53 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h[70102]: 2024-06-02T16:00:53.753+0000 7f9356be7700 -1 mds.h *** got signal Terminated *** 2024-06-02T16:00:55.868 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:55 smithi120 podman[73727]: 2024-06-02 16:00:55.597965405 +0000 UTC m=+1.937882634 container died 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, RELEASE=main-b7328fe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-06-02T16:00:55.868 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:55 smithi120 podman[73727]: 2024-06-02 16:00:55.871106219 +0000 UTC m=+2.211023445 container cleanup 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-02T16:00:56.269 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:55 smithi120 bash[73727]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h 2024-06-02T16:00:56.269 INFO:journalctl@ceph.mds.h.smithi120.stdout:Jun 02 16:00:56 smithi120 podman[73739]: 2024-06-02 16:00:56.113930299 +0000 UTC m=+0.511950701 container remove 2e206c8fc918ba32b8e41221952843acc3ec7cb81d98f6e86b24e0ee2b87b0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-b7328fe) 2024-06-02T16:00:56.291 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-02T16:00:56.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:00:56.322 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-02T16:00:56.322 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-02T16:00:56.322 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.k 2024-06-02T16:00:56.557 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:00:56 smithi120 systemd[1]: Stopping Ceph mds.k for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:00:56.969 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:00:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k[65737]: 2024-06-02T16:00:56.674+0000 7f610cec3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:00:56.969 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:00:56 smithi120 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k[65737]: 2024-06-02T16:00:56.674+0000 7f610cec3700 -1 mds.k *** got signal Terminated *** 2024-06-02T16:01:00.974 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:01:00 smithi120 podman[73848]: 2024-06-02 16:01:00.68842025 +0000 UTC m=+4.146955425 container died 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-b7328fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T16:01:01.298 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:01:01 smithi120 podman[73848]: 2024-06-02 16:01:01.100568645 +0000 UTC m=+4.559103816 container cleanup 6a1084a59bf47f2fcd5dfeaca30bf250a5be8358de27ba5455ccd66020a0a19d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-02T16:01:01.298 INFO:journalctl@ceph.mds.k.smithi120.stdout:Jun 02 16:01:01 smithi120 bash[73848]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-k 2024-06-02T16:01:01.520 DEBUG:teuthology.orchestra.run.smithi120:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-02T16:01:01.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:01:01.551 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-02T16:01:01.551 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-02T16:01:01.551 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.c 2024-06-02T16:01:01.865 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:01 smithi135 systemd[1]: Stopping Ceph mds.c for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:01:02.316 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: 2024-06-02T16:01:01.887+0000 7f41bcab4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:01:02.317 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:01 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c[67129]: 2024-06-02T16:01:01.887+0000 7f41bcab4700 -1 mds.c *** got signal Terminated *** 2024-06-02T16:01:04.909 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:04 smithi135 podman[74151]: 2024-06-02 16:01:04.634692701 +0000 UTC m=+2.878965240 container died ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-06-02T16:01:05.301 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:05 smithi135 podman[74151]: 2024-06-02 16:01:05.000196207 +0000 UTC m=+3.244468741 container cleanup ca383c9ea6a4e7061802fbeef6e4e63d30da81798d3d0efa29eb30b7301aabc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c, RELEASE=main-b7328fe, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T16:01:05.301 INFO:journalctl@ceph.mds.c.smithi135.stdout:Jun 02 16:01:05 smithi135 bash[74151]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-c 2024-06-02T16:01:05.477 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-02T16:01:05.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:01:05.512 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-02T16:01:05.512 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-02T16:01:05.512 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.f 2024-06-02T16:01:05.829 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:05 smithi135 systemd[1]: Stopping Ceph mds.f for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:01:06.316 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f[66823]: 2024-06-02T16:01:05.832+0000 7f9b52c8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:01:06.317 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:05 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f[66823]: 2024-06-02T16:01:05.832+0000 7f9b52c8d700 -1 mds.f *** got signal Terminated *** 2024-06-02T16:01:07.471 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:07 smithi135 podman[74282]: 2024-06-02 16:01:07.152735771 +0000 UTC m=+1.420735435 container died e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f, RELEASE=main-b7328fe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-02T16:01:07.771 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:07 smithi135 podman[74282]: 2024-06-02 16:01:07.54047826 +0000 UTC m=+1.808477878 container cleanup e731c24d4dff6c3df5e029e91c4368516efc29c14b27b1d9b8cd0fc1f0e3bc19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-02T16:01:07.772 INFO:journalctl@ceph.mds.f.smithi135.stdout:Jun 02 16:01:07 smithi135 bash[74282]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-f 2024-06-02T16:01:07.907 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-02T16:01:07.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:01:07.939 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-02T16:01:07.939 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-02T16:01:07.939 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.i 2024-06-02T16:01:08.066 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:08 smithi135 systemd[1]: Stopping Ceph mds.i for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:01:08.566 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: 2024-06-02T16:01:08.260+0000 7f7839588700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:01:08.567 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:08 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i[67448]: 2024-06-02T16:01:08.260+0000 7f7839588700 -1 mds.i *** got signal Terminated *** 2024-06-02T16:01:12.535 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:12 smithi135 podman[74402]: 2024-06-02 16:01:12.265415994 +0000 UTC m=+4.106545687 container died c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-b7328fe, io.buildah.version=1.33.5) 2024-06-02T16:01:12.816 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:12 smithi135 podman[74402]: 2024-06-02 16:01:12.63847789 +0000 UTC m=+4.479607582 container cleanup c30288308b22d6e603add83a33e570c26e5d04f6093a3a1e9b4e41bd17690a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-b7328fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-06-02T16:01:12.817 INFO:journalctl@ceph.mds.i.smithi135.stdout:Jun 02 16:01:12 smithi135 bash[74402]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-i 2024-06-02T16:01:13.063 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-02T16:01:13.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:01:13.094 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-02T16:01:13.095 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-02T16:01:13.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.l 2024-06-02T16:01:13.414 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:13 smithi135 systemd[1]: Stopping Ceph mds.l for 935be7dc-20f4-11ef-bc9b-c7b262605968... 2024-06-02T16:01:13.817 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l[66519]: 2024-06-02T16:01:13.415+0000 7f29b5e94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T16:01:13.817 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:13 smithi135 ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l[66519]: 2024-06-02T16:01:13.415+0000 7f29b5e94700 -1 mds.l *** got signal Terminated *** 2024-06-02T16:01:14.816 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:14 smithi135 podman[74520]: 2024-06-02 16:01:14.493409155 +0000 UTC m=+1.171175563 container died 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-b7328fe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-06-02T16:01:15.316 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:15 smithi135 podman[74520]: 2024-06-02 16:01:15.076101284 +0000 UTC m=+1.753867680 container cleanup 37e4c077cbdec5358491c52d7e47b54e2fd2d52a88a1d50d8dfa1527d5bf57ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d69fb745e0d748b72f6cab1f010a6c2bbc9d2c58cb065d7f63d3528692b7c29a, name=ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=main-b7328fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-02T16:01:15.317 INFO:journalctl@ceph.mds.l.smithi135.stdout:Jun 02 16:01:15 smithi135 bash[74520]: ceph-935be7dc-20f4-11ef-bc9b-c7b262605968-mds-l 2024-06-02T16:01:15.451 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-935be7dc-20f4-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-02T16:01:15.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T16:01:15.481 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-02T16:01:15.482 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T16:01:15.802 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:01:21.212 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T16:01:21.403 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:01:23.829 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T16:01:24.018 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:01:26.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T16:01:26.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T16:01:26.479 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T16:01:26.504 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-02T16:01:26.507 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi090/crash 2024-06-02T16:01:26.508 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T16:01:26.592 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T16:01:26.593 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-06-02T16:01:26.596 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi120/crash 2024-06-02T16:01:26.597 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T16:01:26.660 INFO:teuthology.orchestra.run.smithi120.stderr:tar: /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T16:01:26.660 INFO:teuthology.orchestra.run.smithi120.stderr:tar: Error is not recoverable: exiting now 2024-06-02T16:01:26.662 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi135/crash 2024-06-02T16:01:26.663 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T16:01:26.697 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T16:01:26.697 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-06-02T16:01:26.698 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-02T16:01:26.699 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-02T16:01:26.888 INFO:teuthology.orchestra.run.smithi090.stdout:2024-06-02T15:54:22.886055+0000 mds.b (mds.0) 629 : cluster [ERR] failed to store backtrace on ino 0x10000005515 object, pool 2, errno -2 2024-06-02T16:01:26.889 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-02T16:01:26.890 INFO:tasks.cephadm:Compressing logs... 2024-06-02T16:01:26.890 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T16:01:26.895 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T16:01:26.898 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T16:01:26.921 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T16:01:26.922 INFO:teuthology.orchestra.run.smithi135.stderr:: No such file or directory 2024-06-02T16:01:26.922 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-02T16:01:26.922 INFO:teuthology.orchestra.run.smithi135.stderr:gzip/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-02T16:01:26.922 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T16:01:26.938 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-02T16:01:26.938 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 90.4% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr: 89.6% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr: 94.5% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T16:01:26.939 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-02T16:01:26.940 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-02T16:01:26.940 INFO:teuthology.orchestra.run.smithi135.stderr: 82.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T16:01:26.940 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T16:01:26.940 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.8.log: 93.3% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-02T16:01:26.941 INFO:teuthology.orchestra.run.smithi120.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:: No such file or directory 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-02T16:01:26.942 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr: 90.4% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.10.log: 89.6% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr: 93.3% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-02T16:01:26.943 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T16:01:26.944 INFO:teuthology.orchestra.run.smithi120.stderr: 83.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T16:01:26.944 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-02T16:01:26.945 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-02T16:01:26.946 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-02T16:01:26.951 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T16:01:26.955 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T16:01:26.956 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T16:01:26.958 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T16:01:26.962 INFO:teuthology.orchestra.run.smithi135.stderr: 94.4% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-02T16:01:26.965 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T16:01:26.972 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-02T16:01:26.981 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-02T16:01:26.981 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T16:01:26.982 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T16:01:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T16:01:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-02T16:01:26.996 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-02T16:01:26.997 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-02T16:01:26.998 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-02T16:01:26.999 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-02T16:01:26.999 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-02T16:01:27.001 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-02T16:01:27.002 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T16:01:27.002 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-02T16:01:27.004 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T16:01:27.004 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-02T16:01:27.005 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-02T16:01:27.006 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T16:01:27.007 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-02T16:01:27.050 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 89.6% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T16:01:27.072 INFO:teuthology.orchestra.run.smithi090.stderr: 83.2% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T16:01:27.098 INFO:teuthology.orchestra.run.smithi090.stderr: 90.2% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T16:01:27.189 INFO:teuthology.orchestra.run.smithi090.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-02T16:01:27.320 INFO:teuthology.orchestra.run.smithi090.stderr: 94.5% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-02T16:01:27.394 INFO:teuthology.orchestra.run.smithi090.stderr: 94.3% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-02T16:01:27.510 INFO:teuthology.orchestra.run.smithi090.stderr: 90.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T16:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T16:01:27.547 INFO:teuthology.orchestra.run.smithi135.stderr: 91.7% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-02T16:01:27.648 INFO:teuthology.orchestra.run.smithi120.stderr: 91.6% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-02T16:01:29.332 INFO:teuthology.orchestra.run.smithi120.stderr: 94.7% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-02T16:01:29.460 INFO:teuthology.orchestra.run.smithi090.stderr: 90.5% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-02T16:01:30.066 INFO:teuthology.orchestra.run.smithi135.stderr: 94.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-02T16:01:33.122 INFO:teuthology.orchestra.run.smithi135.stderr: 95.3% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-02T16:01:33.554 INFO:teuthology.orchestra.run.smithi120.stderr: 95.2% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-02T16:01:36.069 INFO:teuthology.orchestra.run.smithi090.stderr: 90.4% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-02T16:01:38.003 INFO:teuthology.orchestra.run.smithi120.stderr: 92.2% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-02T16:01:39.493 INFO:teuthology.orchestra.run.smithi120.stderr: 92.1% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-02T16:01:40.376 INFO:teuthology.orchestra.run.smithi135.stderr: 92.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-02T16:01:40.775 INFO:teuthology.orchestra.run.smithi135.stderr: 92.1% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-02T16:01:41.550 INFO:teuthology.orchestra.run.smithi135.stderr: 92.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-02T16:01:43.405 INFO:teuthology.orchestra.run.smithi120.stderr: 91.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-02T16:01:44.351 INFO:teuthology.orchestra.run.smithi120.stderr: 91.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-02T16:01:46.193 INFO:teuthology.orchestra.run.smithi135.stderr: 91.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-02T16:01:46.195 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-06-02T16:01:46.195 INFO:teuthology.orchestra.run.smithi135.stderr:real 0m19.283s 2024-06-02T16:01:46.195 INFO:teuthology.orchestra.run.smithi135.stderr:user 1m8.853s 2024-06-02T16:01:46.196 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m2.230s 2024-06-02T16:01:49.672 INFO:teuthology.orchestra.run.smithi090.stderr: 92.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-02T16:01:55.103 INFO:teuthology.orchestra.run.smithi090.stderr: 91.9% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-02T16:01:55.293 INFO:teuthology.orchestra.run.smithi090.stderr: 91.7% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-02T16:01:57.789 INFO:teuthology.orchestra.run.smithi090.stderr: 92.0% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-02T16:02:30.365 INFO:teuthology.orchestra.run.smithi120.stderr: 93.5% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-02T16:02:30.367 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-06-02T16:02:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:real 1m3.454s 2024-06-02T16:02:30.367 INFO:teuthology.orchestra.run.smithi120.stderr:user 2m6.621s 2024-06-02T16:02:30.368 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m3.856s 2024-06-02T16:02:34.860 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-02T16:02:46.293 INFO:teuthology.orchestra.run.smithi090.stderr: 93.2% -- replaced with /var/log/ceph/935be7dc-20f4-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-02T16:02:46.298 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-06-02T16:02:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:real 1m19.361s 2024-06-02T16:02:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:user 3m33.029s 2024-06-02T16:02:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m16.194s 2024-06-02T16:02:46.300 INFO:tasks.cephadm:Archiving logs... 2024-06-02T16:02:46.300 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi090/log 2024-06-02T16:02:46.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T16:03:14.796 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi120/log 2024-06-02T16:03:14.798 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T16:03:32.315 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi135/log 2024-06-02T16:03:32.316 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T16:03:42.350 INFO:tasks.cephadm:Removing cluster... 2024-06-02T16:03:42.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force 2024-06-02T16:03:42.667 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:03:43.849 DEBUG:teuthology.orchestra.run.smithi120:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force 2024-06-02T16:03:44.038 INFO:teuthology.orchestra.run.smithi120.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:03:44.443 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 935be7dc-20f4-11ef-bc9b-c7b262605968 --force 2024-06-02T16:03:44.621 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 935be7dc-20f4-11ef-bc9b-c7b262605968 2024-06-02T16:03:44.976 INFO:tasks.cephadm:Removing cephadm ... 2024-06-02T16:03:44.976 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T16:03:45.034 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T16:03:45.049 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T16:03:45.065 INFO:tasks.cephadm:Teardown complete 2024-06-02T16:03:45.066 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-02T16:03:45.080 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-02T16:03:45.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T16:03:45.088 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T16:03:45.092 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T16:03:45.228 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-02T16:03:45.239 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-02T16:03:45.239 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T16:03:45.247 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T16:03:45.250 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T16:03:45.261 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-06-02T16:03:45.263 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-06-02T16:03:45.265 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T16:03:45.266 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-06-02T16:03:45.266 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1015us[ +993us] +/- 55ms 2024-06-02T16:03:45.266 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -599us[ -599us] +/- 73ms 2024-06-02T16:03:45.266 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -909us[ -930us] +/- 74ms 2024-06-02T16:03:45.266 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T16:03:45.267 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T16:03:45.267 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-06-02T16:03:45.267 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1006us[+1001us] +/- 55ms 2024-06-02T16:03:45.268 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -595us[ -599us] +/- 73ms 2024-06-02T16:03:45.268 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -952us[ -956us] +/- 74ms 2024-06-02T16:03:45.268 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T16:03:45.294 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-06-02T16:03:45.303 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T16:03:45.303 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-06-02T16:03:45.304 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1048us[+1042us] +/- 55ms 2024-06-02T16:03:45.304 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -537us[ -543us] +/- 73ms 2024-06-02T16:03:45.304 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 52 -870us[ -877us] +/- 74ms 2024-06-02T16:03:45.304 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T16:03:45.305 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-02T16:03:45.315 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-02T16:03:45.316 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-02T16:03:45.326 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T16:03:45.640 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T16:03:45.708 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T16:03:45.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T16:03:45.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:45.864 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T16:03:45.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:45.893 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T16:03:45.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:45.920 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-02T16:03:45.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-02T16:03:45.945 INFO:teuthology.task.internal:Duration was 2709.724187 seconds 2024-06-02T16:03:45.945 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-02T16:03:45.962 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-02T16:03:45.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T16:03:45.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T16:03:45.973 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T16:03:46.010 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T16:03:46.010 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T16:03:46.153 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T16:03:46.687 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-02T16:03:46.687 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-06-02T16:03:46.687 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T16:03:46.749 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-06-02T16:03:46.749 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T16:03:46.778 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-06-02T16:03:46.778 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T16:03:46.810 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-02T16:03:46.810 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T16:03:46.815 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T16:03:46.820 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T16:03:46.875 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-02T16:03:46.876 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T16:03:46.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T16:03:46.889 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T16:03:47.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-02T16:03:47.998 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-02T16:03:47.998 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T16:03:48.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T16:03:48.097 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T16:03:48.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-02T16:03:48.136 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.143 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.169 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-06-02T16:03:48.192 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-06-02T16:03:48.239 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-06-02T16:03:48.288 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:48.336 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:48.352 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T16:03:48.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T16:03:48.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-02T16:03:48.379 INFO:teuthology.task.internal:Transferring archived files... 2024-06-02T16:03:48.380 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi090 2024-06-02T16:03:48.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T16:03:48.497 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi120 2024-06-02T16:03:48.497 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T16:03:48.534 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-22_20:24:16-fs-main-distro-default-smithi/7720920/remote/smithi135 2024-06-02T16:03:48.534 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T16:03:48.572 INFO:teuthology.task.internal:Removing archive directory... 2024-06-02T16:03:48.572 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T16:03:48.580 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T16:03:48.582 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T16:03:48.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-02T16:03:48.645 INFO:teuthology.task.internal:Not uploading archives. 2024-06-02T16:03:48.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-02T16:03:48.845 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-02T16:03:48.846 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T16:03:48.857 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T16:03:48.861 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T16:03:48.875 INFO:teuthology.orchestra.run.smithi120.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 16:03 /home/ubuntu/cephtest 2024-06-02T16:03:48.879 INFO:teuthology.orchestra.run.smithi135.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 16:03 /home/ubuntu/cephtest 2024-06-02T16:03:48.907 INFO:teuthology.orchestra.run.smithi090.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 16:03 /home/ubuntu/cephtest 2024-06-02T16:03:48.910 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-02T16:03:49.143 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-02T16:03:49.206 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/misc}} duration: 2709.7241866588593 failure_reason: 'Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi090 with status 11: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7aec00165b4b073a25396fd0279151d2b2fa23fe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=afac631631f6485d95c88ad686be3bf4 status: fail success: false 2024-06-02T16:03:49.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-02T16:03:49.283 INFO:teuthology.run:FAIL